Skip to content

[copilot] output validation #117

@pelikhan

Description

@pelikhan

The compiler should add a job to validate the output of the agent.

  • The output is returned by the agentic job as the "output" output variable.
  • the sanitization job should be added after collecting the output. It requires models: read permission only, no write permissions!
  • the type of sanitization should be enabled by default but can be disabled through the output: sanitize: false field.
---
output:
    sanitize: false
---
  • the sanitization strategy is to send the output as is through as a github models call to openai/gpt-4.1-nano, with max-tokens: 10. The output chunk be chunked in groups of 7000 tokens. If the output is malicious, the LLM inference request will be refused by azure. If refused, fail the job.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions