Dependency instructions #167
Conversation
There was a problem hiding this comment.
Pull Request Overview
This pull request introduces a comprehensive set of instruction files for generating dependency risk reports in GitHub Copilot. The contribution splits dependency risk evaluation instructions across multiple layered files to enable distributed ownership between centralized teams (like OSPO) and individual development teams.
- Adds multi-layered instruction files that generate dependency risk reports for packages/libraries
- Implements a hierarchical system where base instructions can be extended by company and repository-specific guidelines
- Updates README.md to include the new dependency risk instructions entry
Reviewed Changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| instructions/dependency-risk/dependency-risk.instructions.md | Main orchestration file that references other dependency risk instruction files |
| instructions/dependency-risk/dependency-risk-repository-level.instructions.md | Repository-specific risk evaluation guidelines and formatting rules |
| instructions/dependency-risk/dependency-risk-company-level.instructions.md | Company-level policies for vulnerability, license, and malicious code detection |
| instructions/dependency-risk/dependency-risk-base-level.instructions.md | Core instructions for data sources, report structure, and risk evaluation criteria |
| instructions/dependency-risk/dependency-risk-README.md | Human-readable documentation explaining the instruction files and their usage |
| README.md | Updated to include dependency risk instructions in the main table |
Comments suppressed due to low confidence (1)
instructions/dependency-risk/dependency-risk-base-level.instructions.md:1
- The link reference
./dependency_instructions_baseLevel.mddoes not match the actual filenamedependency-risk-base-level.instructions.md. This will result in a broken link.
---
instructions/dependency-risk/dependency-risk-repository-level.instructions.md
Outdated
Show resolved
Hide resolved
instructions/dependency-risk/dependency-risk-company-level.instructions.md
Outdated
Show resolved
Hide resolved
instructions/dependency-risk/dependency-risk-company-level.instructions.md
Outdated
Show resolved
Hide resolved
instructions/dependency-risk/dependency-risk-base-level.instructions.md
Outdated
Show resolved
Hide resolved
markdownlit is insisting on exiting other peoples' contributions, but not doing that in this PR
The markdownlit keeps trying to change other people's entries, which is outside scope of this PR.
…ctions.md naming sync Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
…tructions.md naming sync Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
|
Note that the failing check fails because it is expecting a single instructions file, which is not possible in this case. Someone may need to overrule that check. |
removing copilot driven changes to lines not controlled by me
Removed extra |
Removing copilot applied changes that related to my code.
|
Given the design of the PR, we're not going to be able to merge it presently. As you note, the CI job failure is because it expects there to be a 1-1 mapping in the README for the instructions (and prompts/chat modes). This is a requirement of the repo, and bypassing it would result in a permanently broken CI and a higher barrier to contribution, as the One of the primary reasons for the 1-1 mapping is that it removes friction in adding the resources to your own repo, simply click the "Add to VS Code" button. What you're exploring here might be better suited to a "collections" idea, which you can read more about the challenges in this thread. |
Pull Request Checklist
node update-readme.jsand verified thatREADME.mdis up to date.dependency-risk. Also, the README had to be manually edited instead of using the script for same reason.Description
The instruction files in this contribution were created to reduce the risk of developers using packages suggested by Copilot that don't exist and developers using packages suggested by copilot without doing any research into known vulnerabilities, license, or community health issues. These instruction files generate a semi-standardized dependency risk report in the chat window that is easily scannable and understandable by developers such that they make better informed decisions about dependency consumption before writing any code when switching costs are lowest.
Type of Contribution
Additional Notes
The main difference between this contribution and other contributions to this repository is that the instructions for a single Copilot Chat experience are split across more than one file instead of placed in a single file. This was necessary to solve two types of problems with instructions files at once, file staleness and needs for distributed ownership, which are common when certain parts of the instructions reflect repository-specific needs and other parts of the instructions reflect tools, data, and requirements from a centralized team.
Staleness problem
As files that exist in a repository, singular instruction files are immediately stale as soon as they are copied into a given repository from their original source repository. Any changes in the parent require another pull request. If the instruction file is used in hundreds of repositories, this does not scale well and contributes to instructions files being out of date and not reflecting improvements by centralized teams that maybe own the parent.
Distributed ownership of instructions problem
While a MCP can hold an instruction file, this approach does not also allow for any type of per-repository flexibility at the same time. Everyone uses whatever is defined in the MCP server. With dependency risks evaluations, there are types of risks where there is a uniform standard across a company and other types of dependency risks where there is reasonable variability repository to repository as project needs vary. Certain projects may need higher standards. Community health evaluation for small feature complete npm packages and large swiss army knife python packages should reasonably not be using the same thresholds.
Layered instruction files with possibility of partial external instructions solves for both problems
To solve for these problems, the dependency risk report can be generated on the basis of instructions spread across multiple files that are layered. Some of those files can hold repository instruction instructions and vary repository to repository. Other parts of the instructions, like where to source data, what MCP tool endpoints to use, etc, can be put in another instructions file that is in an external repository or MCP server and owned by a centralized team like an OSPO (Open Source Programs Office) or engineering platforms team. A more detailed explanation is in the
instructions/dependency-risk/dependency-risk-README.mdfile.While this approach is here used for generating a prototype dependency risk report, it could be useful any other situation where there is a need for both locally instructions and pulling in externally defined company-wide instructions on the same topic/task.
By submitting this pull request, I confirm that my contribution abides by the Code of Conduct and will be licensed under the MIT License.