feat(py/dotpromptz): test harness that creates suites and test case methods dynamically - #279
Merged
Merged
Conversation
rutujaD1999
approved these changes
May 14, 2025
yesudeep
changed the title
r8zr
reviewed
May 14, 2025
yesudeep
force-pushed
the
yesudeep/feat/thirsty-ring
branch
from
May 14, 2025 09:41
67fe8cc to
c7fbf8e
Compare
…ethods dynamically
The previous implementation of the test harness simply iterated
over each suite and test case within it to run it. It made it hard
to identify and scope failures.
This technique generates test suites and test case methods dynamically
adding them to the module environment before the `unittest.main`
entry-point takes over.
CHANGELOG:
- [ ] Update `spec_test.py` with the newer implementation.
USAGE:
```
zsh❯ uv run spec_test.py
...2025-05-14 02:29:57 [info ] [TEST] history > basic_history > inserts conversation history with proper metadata and roles
.2025-05-14 02:29:57 [info ] [TEST] history > empty_history > handles empty history by only rendering the template content
.2025-05-14 02:29:57 [info ] [TEST] ifEquals > basic > renders false branch when values are not equal
.2025-05-14 02:29:57 [info ] [TEST] ifEquals > basic > renders true branch when values are equal
.2025-05-14 02:29:57 [info ] [TEST] ifEquals > type_safety > treats different types as not equal
.2025-05-14 02:29:57 [info ] [TEST] json > basic > renders json in place
.2025-05-14 02:29:57 [info ] [TEST] json > indented > renders json in place
.2025-05-14 02:29:57 [info ] [TEST] media > basic > renders media part
.2025-05-14 02:29:57 [info ] [TEST] metadata > ext > extension fields are parsed and added to 'ext'
.2025-05-14 02:29:57 [info ] [TEST] metadata > metadata_state > accesses state object from metadata
.2025-05-14 02:29:58 [info ] [TEST] metadata > metadata_state > handles missing state values
.2025-05-14 02:29:58 [info ] [TEST] metadata > metadata_state > handles nested state objects
.2025-05-14 02:29:58 [info ] [TEST] metadata > raw > raw frontmatter is provided on top of parsed frontmatter
.2025-05-14 02:29:58 [info ] [TEST] partials > basic_partial > renders a basic partial
2025-05-14 02:29:58 [debug ] {'event': 'partial_registered', 'name': 'greeting'}
.2025-05-14 02:29:58 [info ] [TEST] partials > nested_resolved_partial > renders a resolver partial inside a resolver partial.
.2025-05-14 02:29:58 [info ] [TEST] partials > partial_with_context > renders a partial with context
2025-05-14 02:29:58 [debug ] {'event': 'partial_registered', 'name': 'userGreeting'}
.2025-05-14 02:29:58 [info ] [TEST] partials > resolved_partial > renders a partial provided by a resolver.
.2025-05-14 02:29:58 [info ] [TEST] picoschema > any_field > returns as expected
.2025-05-14 02:29:58 [info ] [TEST] picoschema > array_of_scalars > returns as expected
.2025-05-14 02:29:58 [info ] [TEST] picoschema > enum_field > returns as expected
.2025-05-14 02:29:58 [info ] [TEST] picoschema > inferred_json_schema_from_properties > returns as expected
.2025-05-14 02:29:58 [info ] [TEST] picoschema > input_and_output > returns as expected
.2025-05-14 02:29:58 [info ] [TEST] picoschema > line_endings_crlf > returns as expected
.2025-05-14 02:29:58 [info ] [TEST] picoschema > named_schema_override_description > returns as expected
.2025-05-14 02:29:58 [info ] [TEST] picoschema > nested_named_schema > returns as expected
.2025-05-14 02:29:58 [info ] [TEST] picoschema > nested_object_in_array_and_out > returns as expected
.2025-05-14 02:29:58 [info ] [TEST] picoschema > required_field > returns as expected
.2025-05-14 02:29:58 [info ] [TEST] picoschema > simple_json_schema_type > returns as expected
.2025-05-14 02:29:58 [info ] [TEST] picoschema > simple_object > returns as expected
.2025-05-14 02:29:58 [info ] [TEST] picoschema > simple_scalar_description > returns as expected
.2025-05-14 02:29:58 [info ] [TEST] picoschema > simple_scalar_description_extra_whitespace > returns as expected
.2025-05-14 02:29:58 [info ] [TEST] picoschema > simple_scalar_description_no_whitespace > returns as expected
.2025-05-14 02:29:58 [info ] [TEST] picoschema > simple_scalar_description_with_commas > returns as expected
.2025-05-14 02:29:58 [info ] [TEST] picoschema > simple_scalar_no_description > returns as expected
.2025-05-14 02:29:58 [info ] [TEST] picoschema > wildcard_fields_with_other_fields > returns as expected
.2025-05-14 02:29:58 [info ] [TEST] picoschema > wildcard_fields_without_other_fields > returns as expected
.2025-05-14 02:29:58 [info ] [TEST] role > all_roles > allows system, user, and model roles
.2025-05-14 02:29:58 [info ] [TEST] role > system_only_prompt > inserts history after system prompt
.2025-05-14 02:29:58 [info ] [TEST] role > system_role > renders variables in system and user role
.2025-05-14 02:29:58 [info ] [TEST] section > basic_section > renders sequential sections with proper metadata and content boundaries
.2025-05-14 02:29:58 [info ] [TEST] section > nested_sections > handles nested and reopened sections with proper metadata boundaries
.2025-05-14 02:29:58 [info ] [TEST] unlessEquals > basic > renders false branch when values are equal
.2025-05-14 02:29:58 [info ] [TEST] unlessEquals > basic > renders true branch when values are different
.2025-05-14 02:29:58 [info ] [TEST] unlessEquals > type_safety > treats different types as not equal
.2025-05-14 02:29:58 [info ] [TEST] variables > basic > does not escape HTML
.2025-05-14 02:29:58 [info ] [TEST] variables > basic > overrides a default variable with a provided variable
.2025-05-14 02:29:58 [info ] [TEST] variables > basic > uses a default variable
.2025-05-14 02:29:58 [info ] [TEST] variables > basic > uses a provided variable
.
----------------------------------------------------------------------
Ran 51 tests in 0.054s
```
yesudeep
force-pushed
the
yesudeep/feat/thirsty-ring
branch
from
May 14, 2025 09:42
c7fbf8e to
0f1fcc3
Compare
github-actions
Bot
added
documentation
yesudeep
added a commit
that referenced
this pull request
Dec 15, 2025
🤖 I have created a release *beep* *boop* --- * add implementation of helpers and util modules; move interfaces into dotpromptz project ([#73](#73)) ([8c7aea1](8c7aea1)) * add stub spec_test.py and script to monitor tests. ([#138](#138)) ([65966e9](65966e9)) * **go/parse:** parse.go implementation [#62](#62) ([#87](#87)) ([d5dc13c](d5dc13c)) * parseDocument python ([#80](#80)) ([82ebc36](82ebc36)) * partial test runner implementation now loads tests ([#139](#139)) ([b09dd2f](b09dd2f)) * Port JS types to Python ([#65](#65)) ([edcb037](edcb037)) * **py/dotprompt:** render and compile implementations ([#263](#263)) ([961d0db](961d0db)) * **py/dotpromptz:** _resolve_metadata for dotprompt ([#226](#226)) ([cfcc87b](cfcc87b)) * **py/dotpromptz:** add initial Dotprompt._resolve_json_schema implementation ([#217](#217)) ([0b62136](0b62136)) * **py/dotpromptz:** add initial Dotprompt._resolve_partials implementation ([#215](#215)) ([03a161c](03a161c)) * **py/dotpromptz:** add initial Dotprompt._resolve_tools implementation and raise ValueError when resolver is None ([#214](#214)) ([57caf5d](57caf5d)) * **py/dotpromptz:** add resolve_json_schema and test case for ResolverCallable that returns an asyncio.Future ([#211](#211)) ([7bbe80d](7bbe80d)) * **py/dotpromptz:** add resolvers module ([#207](#207)) ([826f257](826f257)) * **py/dotpromptz:** because the Picoschema parser can resolve schemas at runtime, we make it an async implementation ([#220](#220)) ([ae285a8](ae285a8)) * **py/dotpromptz:** configure handlebars to not escape by default ([#163](#163)) ([f7c33e1](f7c33e1)) * **py/dotpromptz:** construct test-specific dotprompt instance with partials and partial resolver set up ([#275](#275)) ([0af9a64](0af9a64)) * **py/dotpromptz:** directory-based async and sync implementations of PromptStore and PromptStoreWritable ([#164](#164)) ([ac92fbf](ac92fbf)) * **py/dotpromptz:** Dotprompt.{render|compile} with CompileRenderer stub ([#248](#248)) ([61c578c](61c578c)) * **py/dotpromptz:** implement helpers in terms of the rust implementation of handlebars-rust and fix go flakiness ([#115](#115)) ([314c0b5](314c0b5)) * **py/dotpromptz:** initial bits of Dotprompt class ([#148](#148)) ([90f7838](90f7838)) * **py/dotpromptz:** render_metadata implementation and associated tests ([#229](#229)) ([e66dc45](e66dc45)) * **py/dotpromptz:** simpler spec test runner harness ([#266](#266)) ([89378bf](89378bf)) * **py/dotpromptz:** test harness that creates suites and test case methods dynamically ([#279](#279)) ([1dc2bda](1dc2bda)) * **py/dotpromptz:** translate render_metadata for dotprompt.py from ts ([#227](#227)) ([ae1919b](ae1919b)) * **py/dotpromptz:** use async picoschema_to_json_schema in dotprompt._render_picoschema ([#221](#221)) ([072d95d](072d95d)) * **py:** add SafeString implementation that works with js2py ([#104](#104)) ([1ebeca3](1ebeca3)) * **py:** add support for python 3.14 ([#365](#365)) ([53db722](53db722)) * **py:** fix picoschema spec test ([#303](#303)) ([73439c1](73439c1)) * **py:** implement identify_partials in terms of regexps since we do not have an AST to walk [#90](#90) ([#150](#150)) ([f802275](f802275)) * **py:** implementation of parse.py; refactor parse.ts and update tests. ([#79](#79)) ([47e7245](47e7245)) * **py:** local variables support workaround ([#318](#318)) ([d09598b](d09598b)) * python implementations of helpers ([#129](#129)) ([79c6ef3](79c6ef3)) * python: implement spec test logic ([#285](#285)) ([944f1fe](944f1fe)) * python: picoschema support ISSUE: [#36](#36) CHANGELOG: - [x] Port relevant functionality - [x] Add tests ([#95](#95)) ([0da188c](0da188c)) * **python:** add OpenAI adapter implementation for dotprompt [#38](#38) ([#97](#97)) ([d171f87](d171f87)) * **python:** support lower versions of python (>=3.10) ([#187](#187)) ([4240f9d](4240f9d)) * **py:** utility function to unquote a string literal coming from js2py handlebars helpers ([#107](#107)) ([b3672ca](b3672ca)) * **py:** utility to remove undefined fields from dicts/lists recursively ([#105](#105)) ([d25c911](d25c911)) * use the HEAD version of addlicense ([#280](#280)) ([bdf0d36](bdf0d36)) * change project name for pypi publish ([#200](#200)) ([2c07132](2c07132)) * **docs:** add documentation explaining yaml spec test format and revert typo ([#267](#267)) ([20cd71b](20cd71b)) * **docs:** update docs for helpers.py functions ([#118](#118)) ([40f74d4](40f74d4)) * **docs:** update helper docs ([#132](#132)) ([9b84245](9b84245)) * **go,py:** type fixes and ensure we build/lint the go code in hooks and ci ([#83](#83)) ([19a8257](19a8257)) * **helpers:** use ctx instead of hash to get the fn and inverse ([#131](#131)) ([8749d1f](8749d1f)) * **js:** clean up redundant call to register helpers that have already been registered ([#261](#261)) ([84e3c0c](84e3c0c)) * **license:** use the full license header in source code ([#142](#142)) ([64894ef](64894ef)) * **py/dotpromptz:** add todo about caching resolved schema for later ([#223](#223)) ([728dbaa](728dbaa)) * **py/dotpromptz:** add unit tests for resolver and fix sync resolver error handling ([#208](#208)) ([5e04e28](5e04e28)) * **py/dotpromptz:** address compatibility with python 3.10 and add tox configuration for parallelized tests ([#188](#188)) ([d2ba21f](d2ba21f)) * **py/dotpromptz:** dotprompt.define_tool takes just a tool def ([#224](#224)) ([72039fc](72039fc)) * **py/dotpromptz:** ensure parser handles {CR, CRLF, LF} line endings ([#256](#256)) ([18cff7a](18cff7a)) * **py/dotpromptz:** fix broken picoschema parser tests ([#232](#232)) ([4d154aa](4d154aa)) * **py/dotpromptz:** fix pydantic warnings and tabulate the various types in typing.py ([#196](#196)) ([24dcfdd](24dcfdd)) * **py/dotpromptz:** make resolver concrete in resolve, resolve_tool, and resolve_partial ([#210](#210)) ([3730190](3730190)) * **py/dotpromptz:** pyyaml is a runtime dependency; also update dotpromptz-handlebars to 0.1.3 ([#369](#369)) ([d9132b3](d9132b3)) * **py/dotpromptz:** register initial helpers and partials correctly ([#260](#260)) ([0752865](0752865)) * **py/dotpromptz:** registering partials and start rendering some specs ([#269](#269)) ([aff3c4b](aff3c4b)) * **py/dotpromptz:** remove stray reference to inspect.isasyncgenfunction since we dont deal with those as resolvers ([#216](#216)) ([c300515](c300515)) * **py/dotpromptz:** remove unused Options type from dotprompt.py and fix typo ([#213](#213)) ([ae59431](ae59431)) * **py/dotpromptz:** some lint and add docstrings ([#235](#235)) ([2e1c893](2e1c893)) * **py/handlebarrz:** helpers support in handlebarrz ([#291](#291)) ([d5d66a3](d5d66a3)) * **py/spec_test:** hex-encoded SHA-256 digest rather than base64-encoded SHA-256 digest for module IDs ([#140](#140)) ([796c644](796c644)) * **py:** lint reporting missing docstrings in test files ([#199](#199)) ([4d91514](4d91514)) * **py:** metadata purpose support ([#296](#296)) ([4a0f436](4a0f436)) * remove spurious role type `assistant` ([#169](#169)) ([1b5142c](1b5142c)) * add initial mkdocs documentation for eng [#43](#43) ([#44](#44)) ([31be336](31be336)) * **py/dotpromptz:** document awaitable type hierarchy as ASCII diagram in comments to make code readable ([#212](#212)) ([0926749](0926749)) * **py:** add docstring to parse.py and dotprompt.py modules. ([#246](#246)) ([c9d53bb](c9d53bb)) * **py:** add documentation for the resolvers module ([#209](#209)) ([04b6691](04b6691)) * **py:** ascii diagrams to explain the relationships between the types for easy visualization ([#197](#197)) ([6d775c4](6d775c4)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
yesudeep
added a commit
that referenced
this pull request
Dec 15, 2025
🤖 I have created a release *beep* *boop* --- * add implementation of helpers and util modules; move interfaces into dotpromptz project ([#73](#73)) ([8c7aea1](8c7aea1)) * add stub spec_test.py and script to monitor tests. ([#138](#138)) ([65966e9](65966e9)) * **go/parse:** parse.go implementation [#62](#62) ([#87](#87)) ([d5dc13c](d5dc13c)) * parseDocument python ([#80](#80)) ([82ebc36](82ebc36)) * partial test runner implementation now loads tests ([#139](#139)) ([b09dd2f](b09dd2f)) * Port JS types to Python ([#65](#65)) ([edcb037](edcb037)) * **py/dotprompt:** render and compile implementations ([#263](#263)) ([961d0db](961d0db)) * **py/dotpromptz:** _resolve_metadata for dotprompt ([#226](#226)) ([cfcc87b](cfcc87b)) * **py/dotpromptz:** add initial Dotprompt._resolve_json_schema implementation ([#217](#217)) ([0b62136](0b62136)) * **py/dotpromptz:** add initial Dotprompt._resolve_partials implementation ([#215](#215)) ([03a161c](03a161c)) * **py/dotpromptz:** add initial Dotprompt._resolve_tools implementation and raise ValueError when resolver is None ([#214](#214)) ([57caf5d](57caf5d)) * **py/dotpromptz:** add resolve_json_schema and test case for ResolverCallable that returns an asyncio.Future ([#211](#211)) ([7bbe80d](7bbe80d)) * **py/dotpromptz:** add resolvers module ([#207](#207)) ([826f257](826f257)) * **py/dotpromptz:** because the Picoschema parser can resolve schemas at runtime, we make it an async implementation ([#220](#220)) ([ae285a8](ae285a8)) * **py/dotpromptz:** configure handlebars to not escape by default ([#163](#163)) ([f7c33e1](f7c33e1)) * **py/dotpromptz:** construct test-specific dotprompt instance with partials and partial resolver set up ([#275](#275)) ([0af9a64](0af9a64)) * **py/dotpromptz:** directory-based async and sync implementations of PromptStore and PromptStoreWritable ([#164](#164)) ([ac92fbf](ac92fbf)) * **py/dotpromptz:** Dotprompt.{render|compile} with CompileRenderer stub ([#248](#248)) ([61c578c](61c578c)) * **py/dotpromptz:** implement helpers in terms of the rust implementation of handlebars-rust and fix go flakiness ([#115](#115)) ([314c0b5](314c0b5)) * **py/dotpromptz:** initial bits of Dotprompt class ([#148](#148)) ([90f7838](90f7838)) * **py/dotpromptz:** render_metadata implementation and associated tests ([#229](#229)) ([e66dc45](e66dc45)) * **py/dotpromptz:** simpler spec test runner harness ([#266](#266)) ([89378bf](89378bf)) * **py/dotpromptz:** test harness that creates suites and test case methods dynamically ([#279](#279)) ([1dc2bda](1dc2bda)) * **py/dotpromptz:** translate render_metadata for dotprompt.py from ts ([#227](#227)) ([ae1919b](ae1919b)) * **py/dotpromptz:** use async picoschema_to_json_schema in dotprompt._render_picoschema ([#221](#221)) ([072d95d](072d95d)) * **py:** add SafeString implementation that works with js2py ([#104](#104)) ([1ebeca3](1ebeca3)) * **py:** add support for python 3.14 ([#365](#365)) ([53db722](53db722)) * **py:** fix picoschema spec test ([#303](#303)) ([73439c1](73439c1)) * **py:** implement identify_partials in terms of regexps since we do not have an AST to walk [#90](#90) ([#150](#150)) ([f802275](f802275)) * **py:** implementation of parse.py; refactor parse.ts and update tests. ([#79](#79)) ([47e7245](47e7245)) * **py:** local variables support workaround ([#318](#318)) ([d09598b](d09598b)) * python implementations of helpers ([#129](#129)) ([79c6ef3](79c6ef3)) * python: implement spec test logic ([#285](#285)) ([944f1fe](944f1fe)) * python: picoschema support ISSUE: [#36](#36) CHANGELOG: - [x] Port relevant functionality - [x] Add tests ([#95](#95)) ([0da188c](0da188c)) * **python:** add OpenAI adapter implementation for dotprompt [#38](#38) ([#97](#97)) ([d171f87](d171f87)) * **python:** support lower versions of python (>=3.10) ([#187](#187)) ([4240f9d](4240f9d)) * **py:** utility function to unquote a string literal coming from js2py handlebars helpers ([#107](#107)) ([b3672ca](b3672ca)) * **py:** utility to remove undefined fields from dicts/lists recursively ([#105](#105)) ([d25c911](d25c911)) * use the HEAD version of addlicense ([#280](#280)) ([bdf0d36](bdf0d36)) * change project name for pypi publish ([#200](#200)) ([2c07132](2c07132)) * **docs:** add documentation explaining yaml spec test format and revert typo ([#267](#267)) ([20cd71b](20cd71b)) * **docs:** update docs for helpers.py functions ([#118](#118)) ([40f74d4](40f74d4)) * **docs:** update helper docs ([#132](#132)) ([9b84245](9b84245)) * **go,py:** type fixes and ensure we build/lint the go code in hooks and ci ([#83](#83)) ([19a8257](19a8257)) * **helpers:** use ctx instead of hash to get the fn and inverse ([#131](#131)) ([8749d1f](8749d1f)) * **js:** clean up redundant call to register helpers that have already been registered ([#261](#261)) ([84e3c0c](84e3c0c)) * **license:** use the full license header in source code ([#142](#142)) ([64894ef](64894ef)) * **py/dotpromptz:** add todo about caching resolved schema for later ([#223](#223)) ([728dbaa](728dbaa)) * **py/dotpromptz:** add unit tests for resolver and fix sync resolver error handling ([#208](#208)) ([5e04e28](5e04e28)) * **py/dotpromptz:** address compatibility with python 3.10 and add tox configuration for parallelized tests ([#188](#188)) ([d2ba21f](d2ba21f)) * **py/dotpromptz:** dotprompt.define_tool takes just a tool def ([#224](#224)) ([72039fc](72039fc)) * **py/dotpromptz:** ensure parser handles {CR, CRLF, LF} line endings ([#256](#256)) ([18cff7a](18cff7a)) * **py/dotpromptz:** fix broken picoschema parser tests ([#232](#232)) ([4d154aa](4d154aa)) * **py/dotpromptz:** fix pydantic warnings and tabulate the various types in typing.py ([#196](#196)) ([24dcfdd](24dcfdd)) * **py/dotpromptz:** make resolver concrete in resolve, resolve_tool, and resolve_partial ([#210](#210)) ([3730190](3730190)) * **py/dotpromptz:** pyyaml is a runtime dependency; also update dotpromptz-handlebars to 0.1.3 ([#369](#369)) ([d9132b3](d9132b3)) * **py/dotpromptz:** register initial helpers and partials correctly ([#260](#260)) ([0752865](0752865)) * **py/dotpromptz:** registering partials and start rendering some specs ([#269](#269)) ([aff3c4b](aff3c4b)) * **py/dotpromptz:** remove stray reference to inspect.isasyncgenfunction since we dont deal with those as resolvers ([#216](#216)) ([c300515](c300515)) * **py/dotpromptz:** remove unused Options type from dotprompt.py and fix typo ([#213](#213)) ([ae59431](ae59431)) * **py/dotpromptz:** some lint and add docstrings ([#235](#235)) ([2e1c893](2e1c893)) * **py/handlebarrz:** helpers support in handlebarrz ([#291](#291)) ([d5d66a3](d5d66a3)) * **py/spec_test:** hex-encoded SHA-256 digest rather than base64-encoded SHA-256 digest for module IDs ([#140](#140)) ([796c644](796c644)) * **py:** lint reporting missing docstrings in test files ([#199](#199)) ([4d91514](4d91514)) * **py:** metadata purpose support ([#296](#296)) ([4a0f436](4a0f436)) * remove spurious role type `assistant` ([#169](#169)) ([1b5142c](1b5142c)) * add initial mkdocs documentation for eng [#43](#43) ([#44](#44)) ([31be336](31be336)) * **py/dotpromptz:** document awaitable type hierarchy as ASCII diagram in comments to make code readable ([#212](#212)) ([0926749](0926749)) * **py:** add docstring to parse.py and dotprompt.py modules. ([#246](#246)) ([c9d53bb](c9d53bb)) * **py:** add documentation for the resolvers module ([#209](#209)) ([04b6691](04b6691)) * **py:** ascii diagrams to explain the relationships between the types for easy visualization ([#197](#197)) ([6d775c4](6d775c4)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
yesudeep
added a commit
that referenced
this pull request
Dec 15, 2025
🤖 I have created a release *beep* *boop* --- * add implementation of helpers and util modules; move interfaces into dotpromptz project ([#73](#73)) ([8c7aea1](8c7aea1)) * add stub spec_test.py and script to monitor tests. ([#138](#138)) ([65966e9](65966e9)) * **go/parse:** parse.go implementation [#62](#62) ([#87](#87)) ([d5dc13c](d5dc13c)) * parseDocument python ([#80](#80)) ([82ebc36](82ebc36)) * partial test runner implementation now loads tests ([#139](#139)) ([b09dd2f](b09dd2f)) * Port JS types to Python ([#65](#65)) ([edcb037](edcb037)) * **py/dotprompt:** render and compile implementations ([#263](#263)) ([961d0db](961d0db)) * **py/dotpromptz:** _resolve_metadata for dotprompt ([#226](#226)) ([cfcc87b](cfcc87b)) * **py/dotpromptz:** add initial Dotprompt._resolve_json_schema implementation ([#217](#217)) ([0b62136](0b62136)) * **py/dotpromptz:** add initial Dotprompt._resolve_partials implementation ([#215](#215)) ([03a161c](03a161c)) * **py/dotpromptz:** add initial Dotprompt._resolve_tools implementation and raise ValueError when resolver is None ([#214](#214)) ([57caf5d](57caf5d)) * **py/dotpromptz:** add resolve_json_schema and test case for ResolverCallable that returns an asyncio.Future ([#211](#211)) ([7bbe80d](7bbe80d)) * **py/dotpromptz:** add resolvers module ([#207](#207)) ([826f257](826f257)) * **py/dotpromptz:** because the Picoschema parser can resolve schemas at runtime, we make it an async implementation ([#220](#220)) ([ae285a8](ae285a8)) * **py/dotpromptz:** configure handlebars to not escape by default ([#163](#163)) ([f7c33e1](f7c33e1)) * **py/dotpromptz:** construct test-specific dotprompt instance with partials and partial resolver set up ([#275](#275)) ([0af9a64](0af9a64)) * **py/dotpromptz:** directory-based async and sync implementations of PromptStore and PromptStoreWritable ([#164](#164)) ([ac92fbf](ac92fbf)) * **py/dotpromptz:** Dotprompt.{render|compile} with CompileRenderer stub ([#248](#248)) ([61c578c](61c578c)) * **py/dotpromptz:** implement helpers in terms of the rust implementation of handlebars-rust and fix go flakiness ([#115](#115)) ([314c0b5](314c0b5)) * **py/dotpromptz:** initial bits of Dotprompt class ([#148](#148)) ([90f7838](90f7838)) * **py/dotpromptz:** render_metadata implementation and associated tests ([#229](#229)) ([e66dc45](e66dc45)) * **py/dotpromptz:** simpler spec test runner harness ([#266](#266)) ([89378bf](89378bf)) * **py/dotpromptz:** test harness that creates suites and test case methods dynamically ([#279](#279)) ([1dc2bda](1dc2bda)) * **py/dotpromptz:** translate render_metadata for dotprompt.py from ts ([#227](#227)) ([ae1919b](ae1919b)) * **py/dotpromptz:** use async picoschema_to_json_schema in dotprompt._render_picoschema ([#221](#221)) ([072d95d](072d95d)) * **py:** add SafeString implementation that works with js2py ([#104](#104)) ([1ebeca3](1ebeca3)) * **py:** add support for python 3.14 ([#365](#365)) ([53db722](53db722)) * **py:** fix picoschema spec test ([#303](#303)) ([73439c1](73439c1)) * **py:** implement identify_partials in terms of regexps since we do not have an AST to walk [#90](#90) ([#150](#150)) ([f802275](f802275)) * **py:** implementation of parse.py; refactor parse.ts and update tests. ([#79](#79)) ([47e7245](47e7245)) * **py:** local variables support workaround ([#318](#318)) ([d09598b](d09598b)) * python implementations of helpers ([#129](#129)) ([79c6ef3](79c6ef3)) * python: implement spec test logic ([#285](#285)) ([944f1fe](944f1fe)) * python: picoschema support ISSUE: [#36](#36) CHANGELOG: - [x] Port relevant functionality - [x] Add tests ([#95](#95)) ([0da188c](0da188c)) * **python:** add OpenAI adapter implementation for dotprompt [#38](#38) ([#97](#97)) ([d171f87](d171f87)) * **python:** support lower versions of python (>=3.10) ([#187](#187)) ([4240f9d](4240f9d)) * **py:** utility function to unquote a string literal coming from js2py handlebars helpers ([#107](#107)) ([b3672ca](b3672ca)) * **py:** utility to remove undefined fields from dicts/lists recursively ([#105](#105)) ([d25c911](d25c911)) * use the HEAD version of addlicense ([#280](#280)) ([bdf0d36](bdf0d36)) * change project name for pypi publish ([#200](#200)) ([2c07132](2c07132)) * **docs:** add documentation explaining yaml spec test format and revert typo ([#267](#267)) ([20cd71b](20cd71b)) * **docs:** update docs for helpers.py functions ([#118](#118)) ([40f74d4](40f74d4)) * **docs:** update helper docs ([#132](#132)) ([9b84245](9b84245)) * **go,py:** type fixes and ensure we build/lint the go code in hooks and ci ([#83](#83)) ([19a8257](19a8257)) * **helpers:** use ctx instead of hash to get the fn and inverse ([#131](#131)) ([8749d1f](8749d1f)) * **js:** clean up redundant call to register helpers that have already been registered ([#261](#261)) ([84e3c0c](84e3c0c)) * **license:** use the full license header in source code ([#142](#142)) ([64894ef](64894ef)) * **py/dotpromptz:** add todo about caching resolved schema for later ([#223](#223)) ([728dbaa](728dbaa)) * **py/dotpromptz:** add unit tests for resolver and fix sync resolver error handling ([#208](#208)) ([5e04e28](5e04e28)) * **py/dotpromptz:** address compatibility with python 3.10 and add tox configuration for parallelized tests ([#188](#188)) ([d2ba21f](d2ba21f)) * **py/dotpromptz:** dotprompt.define_tool takes just a tool def ([#224](#224)) ([72039fc](72039fc)) * **py/dotpromptz:** ensure parser handles {CR, CRLF, LF} line endings ([#256](#256)) ([18cff7a](18cff7a)) * **py/dotpromptz:** fix broken picoschema parser tests ([#232](#232)) ([4d154aa](4d154aa)) * **py/dotpromptz:** fix pydantic warnings and tabulate the various types in typing.py ([#196](#196)) ([24dcfdd](24dcfdd)) * **py/dotpromptz:** make resolver concrete in resolve, resolve_tool, and resolve_partial ([#210](#210)) ([3730190](3730190)) * **py/dotpromptz:** pyyaml is a runtime dependency; also update dotpromptz-handlebars to 0.1.3 ([#369](#369)) ([d9132b3](d9132b3)) * **py/dotpromptz:** register initial helpers and partials correctly ([#260](#260)) ([0752865](0752865)) * **py/dotpromptz:** registering partials and start rendering some specs ([#269](#269)) ([aff3c4b](aff3c4b)) * **py/dotpromptz:** remove stray reference to inspect.isasyncgenfunction since we dont deal with those as resolvers ([#216](#216)) ([c300515](c300515)) * **py/dotpromptz:** remove unused Options type from dotprompt.py and fix typo ([#213](#213)) ([ae59431](ae59431)) * **py/dotpromptz:** some lint and add docstrings ([#235](#235)) ([2e1c893](2e1c893)) * **py/handlebarrz:** helpers support in handlebarrz ([#291](#291)) ([d5d66a3](d5d66a3)) * **py/spec_test:** hex-encoded SHA-256 digest rather than base64-encoded SHA-256 digest for module IDs ([#140](#140)) ([796c644](796c644)) * **py:** lint reporting missing docstrings in test files ([#199](#199)) ([4d91514](4d91514)) * **py:** metadata purpose support ([#296](#296)) ([4a0f436](4a0f436)) * remove spurious role type `assistant` ([#169](#169)) ([1b5142c](1b5142c)) * add initial mkdocs documentation for eng [#43](#43) ([#44](#44)) ([31be336](31be336)) * **py/dotpromptz:** document awaitable type hierarchy as ASCII diagram in comments to make code readable ([#212](#212)) ([0926749](0926749)) * **py:** add docstring to parse.py and dotprompt.py modules. ([#246](#246)) ([c9d53bb](c9d53bb)) * **py:** add documentation for the resolvers module ([#209](#209)) ([04b6691](04b6691)) * **py:** ascii diagrams to explain the relationships between the types for easy visualization ([#197](#197)) ([6d775c4](6d775c4)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The previous implementation of the test harness simply iterated
over each suite and test case within it to run it. It made it hard
to identify and scope failures.
This technique generates test suites and test case methods dynamically
adding them to the module environment before the
unittest.mainentry-point takes over.
CHANGELOG:
spec_test.pywith the newer implementation.USAGE: