Skip to content

Refine evaluation process #4

Description

@cyyeh

In the current evaluation process, there are some issues for further refinement

  • reproducibility: there is no dvc.lock related file to make sure the pipelines are reproducible in evaluation.(we can directly reference this to spider-benchmark repo; however, if it's not that important as of now, we can skip this right now.)
  • evaluation report:
    • adding cost/latency
    • adding pipeline metadata so that we can easily understand what parameters cause the differences among different reports

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions