8000 Create a pool of question/answers that can be used to evaluate the system · Issue #4 · focusconsulting/ai-legislation · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content
Create a pool of question/answers that can be used to evaluate the system #4
Open
@mkalish

Description

@mkalish

Description

In order to evaluate the efficacy of the system, there will need to be a pool of curated questions and answers that can be used to test the system.

Acceptance Criteria

  • 200 question/answer pairs for the AI legislation that the system will work with
    • Some questions should be specific to individual legislation
    • Some questions should involve comparing multiple legislation
    • Some questions should involve asking about specific state(s)

Developer notes

LLMs should be leveraged in order to rapidly generate the question answer pairs

  • Summarizing large blocks of texts into an answer
  • Scanning documents to generate questions

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions

      0