Skip to content

Conversation

Copilot
Copy link
Contributor

@Copilot Copilot AI commented Aug 20, 2025

This PR implements automatic upload of the GITHUB_AW_OUTPUT file as a workflow artifact named aw_output.txt in all generated agentic workflows, with validation to only upload when the output file contains non-empty content. Additionally, the logs command is now aware of and can properly handle this new artifact.

Problem

Previously, the agentic output file (GITHUB_AW_OUTPUT) was created during workflow execution and its contents were set as a workflow output, but the file itself was not persisted as an artifact. This made it difficult to:

  • Download and inspect the agent's raw output for debugging
  • Enable offline analysis or replay of agentic outputs
  • Provide reproducibility for integration tests or end-to-end evaluations

Additionally, there was no validation to prevent uploading empty or non-existent output files as artifacts, and the logs command was not aware of this new artifact type.

Solution

Artifact Upload Implementation

Modified the generateOutputCollectionStep function in pkg/workflow/compiler.go to add an artifact upload step that:

  • Uses actions/upload-artifact@v4 to upload the output file
  • Names the artifact aw_output.txt for consistent identification
  • Uses if: always() && steps.collect_output.outputs.output != '' to ensure upload happens even if previous steps fail, but only when the collection step found non-empty content
  • Sets if-no-files-found: warn to handle edge cases where the output file doesn't exist
  • Uploads the file referenced by ${{ env.GITHUB_AW_OUTPUT }}

Logs Command Awareness

Enhanced the logs command to be fully aware of the new artifact:

  • Detection and Reporting: In verbose mode, specifically detects and reports the presence of aw_output.txt files with human-readable file size information
  • Documentation: Updated help text to clearly document all available artifacts including the new agent output file
  • Processing: The artifact is automatically downloaded and processed since it uses the .txt extension
  • Utilities: Added formatFileSize helper function for consistent file size formatting

Changes

  • pkg/workflow/compiler.go: Added artifact upload step to generateOutputCollectionStep function with non-empty content validation
  • pkg/workflow/agentic_output_test.go: Updated test to verify the new upload step presence, correct step ordering, and non-empty content validation
  • pkg/cli/logs.go: Enhanced to detect, report, and document the new aw_output.txt artifact
  • pkg/cli/logs_test.go: Added comprehensive tests for artifact detection and file size formatting
  • Generated workflows: All existing workflow lock files were recompiled to include the new artifact upload step with validation

Testing

  • All existing tests continue to pass
  • New test assertions verify the upload step is present with correct configuration and validation condition
  • Added tests for artifact detection in the logs command and file size formatting utilities
  • Manual workflow compilation confirms the expected artifact upload step is generated with proper non-empty content checking
  • Step ordering is validated: Setup → Execute → Collect → Upload → Other uploads
  • Logs command properly detects and reports on the new artifact type

Fixes #126.


✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
@Copilot Copilot AI changed the title [WIP] [copilot] Compiler: Upload GITHUB_AW_OUTPUT as workflow artifact "aw_output.txt" Add GITHUB_AW_OUTPUT artifact upload to workflow compiler Aug 20, 2025
@Copilot Copilot AI requested a review from pelikhan August 20, 2025 01:23
Copilot finished work on behalf of pelikhan August 20, 2025 01:23
@@ -2322,6 +2322,13 @@ func (c *Compiler) generateOutputCollectionStep(yaml *strings.Builder, data *Wor
yaml.WriteString(" echo '``````markdown' >> $GITHUB_STEP_SUMMARY\n")
yaml.WriteString(" cat ${{ env.GITHUB_AW_OUTPUT }} >> $GITHUB_STEP_SUMMARY\n")
yaml.WriteString(" echo '``````' >> $GITHUB_STEP_SUMMARY\n")
yaml.WriteString(" - name: Upload agentic output file\n")
yaml.WriteString(" if: always()\n")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot add a check for a non-empty output file if possible

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added a check for non-empty output file. The upload step now uses if: always() && steps.collect_output.outputs.output != '' which only uploads the artifact when the collection step found non-empty content. This reuses the existing validation logic that checks for file existence and empty content. See commit c01a83d.

- Modified upload step condition from `if: always()` to `if: always() && steps.collect_output.outputs.output != ''`
- Only uploads artifact when the collection step found non-empty content
- Updated test to verify the new condition is present
- Recompiled all existing workflow lock files with the new condition

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Aug 20, 2025

Just as a heads up, I was blocked by some firewall rules while working on your feedback. Expand below for details.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • api.contoso.com
    • Triggering command: /tmp/go-build315353999/b278/cli.test -test.testlogfile=/tmp/go-build315353999/b278/testlog.txt -test.paniconexit0 -test.timeout=10m0s -test.v=true (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

@Copilot Copilot AI changed the title Add GITHUB_AW_OUTPUT artifact upload to workflow compiler Add GITHUB_AW_OUTPUT artifact upload with non-empty file validation Aug 20, 2025
@Copilot Copilot AI requested a review from pelikhan August 20, 2025 01:46
Copilot finished work on behalf of pelikhan August 20, 2025 01:46
@pelikhan pelikhan marked this pull request as ready for review August 20, 2025 01:49
@pelikhan
Copy link
Contributor

@copilot make sure the logs command is aware of this new artifact file.

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Aug 20, 2025

@copilot make sure the logs command is aware of this new artifact file.

The logs command is now aware of the new aw_output.txt artifact. The changes include:

  • Specific detection and reporting of aw_output.txt file in verbose mode with file size information
  • Updated help text to document all available artifacts including the new agent output file
  • Existing functionality automatically downloads and processes the artifact since it's a .txt file
  • Added comprehensive tests and file size formatting utilities

See commit 9930c9d.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • api.contoso.com
    • Triggering command: /tmp/go-build765531342/b278/cli.test -test.testlogfile=/tmp/go-build765531342/b278/testlog.txt -test.paniconexit0 -test.timeout=10m0s -test.v=true (dns block)
    • Triggering command: /tmp/go-build3741465731/b278/cli.test -test.testlogfile=/tmp/go-build3741465731/b278/testlog.txt -test.paniconexit0 -test.timeout=10m0s -test.v=true (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

@Copilot Copilot AI changed the title Add GITHUB_AW_OUTPUT artifact upload with non-empty file validation Add GITHUB_AW_OUTPUT artifact upload with non-empty file validation and logs command awareness Aug 20, 2025
@Copilot Copilot AI requested a review from pelikhan August 20, 2025 02:00
Copilot finished work on behalf of pelikhan August 20, 2025 02:00
@pelikhan pelikhan merged commit 3d0f6d1 into main Aug 20, 2025
6 checks passed
@pelikhan pelikhan deleted the copilot/fix-126 branch August 20, 2025 05:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[copilot] Compiler: Upload GITHUB_AW_OUTPUT as workflow artifact "aw_output.txt"
2 participants