Prepare for deployment#2
Merged
Merged
Conversation
dimitri-yatsenko
approved these changes
Sep 19, 2022
Ernaldis
approved these changes
Sep 19, 2022
guzman-raphael
pushed a commit
that referenced
this pull request
Jun 8, 2023
wf triger branch master->main
dimitri-yatsenko
added a commit
that referenced
this pull request
Jan 14, 2026
Resolves contradictory version messaging identified in cohesion review: - installation.md said '2.0 in preparation' but was ambiguous - versioning.md treats 2.0 as baseline (correct) - Users confused about which version to install Changes: - Restructured installation.md with clear pre-release vs stable sections - Added decision table for version mismatch scenarios - Updated landing page (index.md) to clearly state pre-release status - Provides explicit instructions for both testing 2.0 and using stable 0.14.x - Regenerated llms-full.txt Impact: - New users can now make informed decision: test 2.0 vs use stable - Clear paths to legacy docs, migration guide, or pre-release installation - Eliminates contradiction between installation and versioning pages Fixes: COHESION-REVIEW.md Issue #2 (High Priority)
4 tasks
4 tasks
dimitri-yatsenko
added a commit
that referenced
this pull request
Jun 5, 2026
Fixes for items #2 and #4 from her review: - storage-adapter-api.md: removed the "Built-in adapters" section and the claim that file/s3/gcs/azure protocols are themselves StorageAdapter subclasses. In v2.2.3 those protocols are still served by the existing StorageBackend._create_filesystem() dispatch; migrating them onto the public adapter contract is tracked separately. - storage-adapter-api.md: added an "Available in datajoint ≥ 2.2.4" note near the StorageAdapter base-class introduction. dj.StorageAdapter / dj.get_storage_adapter() are not exported in 2.2.3; a paired datajoint-python PR adds the exports for 2.2.4. - whats-new-2-2-3.md: reframed the "Storage-adapter plugin contract" section to match — the contract is for third-party adapters; built-ins continue via internal dispatch. - manage-secrets.md: corrected the lone remaining DJ_TLS=true → DJ_USE_TLS=true in the env-var example (line 88). Items #3 (settings.py "New in 2.3" markers) and #1 (StorageAdapter export) are addressed by separate datajoint-python PRs.
4 tasks
dimitri-yatsenko
added a commit
that referenced
this pull request
Jul 2, 2026
Per the decision to document rather than close the two read-gate coverage gaps Thinh flagged on datajoint/datajoint-python#1474: - Existence/count idioms (len/bool/in) issue COUNT/EXISTS SQL directly, not through the gated fetch path, so reads of undeclared tables via those are not caught. - Restriction-by-table (Ancestor & Undeclared) renders as an IN (subquery) the gate's base-table analysis does not inspect. Both are now listed under 'Enforcement model and its limits' as best-effort client-side gaps. Comprehensive enforcement is the platform's job: its code-deployment CI/CD combines these runtime provenance checks with agentic review of the make() source to cover the paths the runtime guardrail cannot.
dimitri-yatsenko
added a commit
that referenced
this pull request
Jul 2, 2026
…venance (#183) * docs(provenance): spec for Diagram.trace + self.upstream + strict_provenance Detailed normative specification for the canonical provenance trinity landing in DataJoint 2.3: - Diagram.trace(table_expr) — upstream mirror of Diagram.cascade(), walks the FK graph from a restricted seed to every ancestor with OR convergence. Reuses the upward propagation rules (U1/U2/U3) defined in the Cascade Specification. Supports trace[TableClass] returning a pre-restricted QueryExpression, and trace[str] returning a FreeTable. - self.upstream — property on AutoPopulate that the framework sets to Diagram.trace(self & key) before each make() invocation. Pre-restricted ancestor access for ergonomic, provenance-safe reads. - dj.config["strict_provenance"] — runtime flag (default False). When True, gates make() so reads must go through self.upstream and writes must target self or self's Parts with key-consistent primary keys. Spec structure: - Why this exists — the convention/enforcement gap, why three pieces. - Concepts — trace as cascade's mirror, OR convergence, the make() provenance boundary, why upward rules are shared with cascade. - §1 Diagram.trace — signature, behavior, indexing (class + string), counts() / heading() / iter, worked examples. - §2 self.upstream — construction lifecycle, allowed table set, per-key behavior, examples. - §3 strict_provenance — config key, read enforcement table, write enforcement table, key-consistency rule, opt-in rationale, examples of compliant + violating make() bodies. - Integration — end-to-end strict-mode example, properties guaranteed. - Migration path — staged adoption from on-default-off to enforcement. - Out of scope — static analysis, default flip, downstream provenance metadata persistence. Examples use core DataJoint types (int32, float64, longblob) per project convention. Cross-links to cascade.md (shared upward rules), autopopulate.md, diagram.md, entity-integrity.md. Nav entry under Reference > Specifications > Data Operations between AutoPopulate and Job Metadata. Slated for DataJoint 2.3. T2.2.a (#1423), T2.2.b (#1424), T2.2.c (#1425) implementations land against this spec. * docs(provenance): reframe strict_provenance as best-effort guardrail; align spec with shipped code Reviewer reconciliation (ttngu207, MilagrosMarin) against the three feature branches, plus a reframing of the enforcement model: - strict_provenance: describe the runtime check honestly as a best-effort, client-side development guardrail, not an airtight boundary. Raw SQL, other clients, and direct DB access bypass it; comprehensive enforcement is a code-inspection problem (static where sound, agentic on the platform) and is positioned as the roadmap. Softened 'runtime guarantee' / 'provenance-safe by construction' language throughout. - Read enforcement: corrected the table + violating example — a direct read of a declared ancestor is Allowed in 2.3 (only undeclared tables are blocked), matching assert_read_allowed. Added a note on what the check does/doesn't distinguish. Error messages now match provenance.py. - trace.counts(): keyed by full table name (dict[str,int]), includes the seed. - Dropped trace.heading() (not implemented). - iter(trace): parents/roots first (topological), not 'deepest first'. - Concurrency: contextvars.ContextVar, not thread-local; removed the dangling 'see Implementation notes' reference. - Noted load_all_upstream is new; glossed make_kwargs; documented self.upstream non-caching; narrowed the ancestor-Parts allowed-set wording. * docs(provenance): attribute code inspection to platform code-deployment CI/CD Draw the boundary plainly: the open-source core ships only the runtime guardrail; comprehensive code inspection (static + agentic) is performed by the DataJoint platform's code-deployment CI/CD process, not the OSS framework. Applied consistently across the intro, the enforcement-model subsection, the integration properties, the migration path, and the out-of-scope list. * docs(provenance): document read-gate best-effort limits (#2/#3) Per the decision to document rather than close the two read-gate coverage gaps Thinh flagged on datajoint/datajoint-python#1474: - Existence/count idioms (len/bool/in) issue COUNT/EXISTS SQL directly, not through the gated fetch path, so reads of undeclared tables via those are not caught. - Restriction-by-table (Ancestor & Undeclared) renders as an IN (subquery) the gate's base-table analysis does not inspect. Both are now listed under 'Enforcement model and its limits' as best-effort client-side gaps. Comprehensive enforcement is the platform's job: its code-deployment CI/CD combines these runtime provenance checks with agentic review of the make() source to cover the paths the runtime guardrail cannot. * docs(provenance): cohesion/completeness pass now that the trinity has shipped - References: 'to land in 2.3' -> shipped; link the merged PRs (#1471/#1473/#1474) and the #1468 cascade rules they build on. - trace[T]: correct the return type — both the class-form and string-form return a FreeTable (a QueryExpression); the class form does NOT return a restricted class instance (matches shipped __getitem__). - Intro: drop the misleading 'checked default' phrasing (the flag defaults to off / opt-in), reword to 'opt-in runtime check'. - Key consistency: note the check applies to dict-style rows; positional/numpy rows carry no names to check and pass unchecked (matches _check_row_key).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.