Skip to content

Implement a File Link Resolver #5981

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 10 commits into from
Jun 10, 2025
Merged

Conversation

incrypto32
Copy link
Member

This PR implements a file system based link resolver aimed to replace IpfsLinkResolver for local development, this is in preparation for #5977

@incrypto32 incrypto32 changed the base branch from master to krishna/refactor-main May 1, 2025 13:02
@incrypto32 incrypto32 self-assigned this May 1, 2025
@lutter lutter self-requested a review May 1, 2025 15:54
@incrypto32 incrypto32 force-pushed the krishna/refactor-main branch from c8b8169 to 15fe3c3 Compare May 6, 2025 10:38
@incrypto32 incrypto32 force-pushed the krishna/file-link-resolver branch from 3ef16d5 to 23e91f1 Compare May 6, 2025 12:59

async fn json_stream(&self, _logger: &Logger, _link: &Link) -> Result<JsonValueStream, Error> {
Err(anyhow!("json_stream is not implemented for FileLinkResolver").into())
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Making these always return errors is sorta ugly. It would be nicer to have a trait FileResolver that doesn't have these two and a trait LinkResolver that does and change the code to use the right one. That's going to be a bigger change, so fine to do it in a separate PR, but we shouldn't let this linger for too long.

The danger with leaving this too long is that the code becomes brittle since now every user of LinkResolver needs to make sure it gets the right kind as that's not guaranteed by the type alone anymore

@incrypto32 incrypto32 force-pushed the krishna/file-link-resolver branch 6 times, most recently from f7e1520 to cd2d014 Compare May 9, 2025 08:48
Base automatically changed from krishna/refactor-main to krishna/gnd-base June 10, 2025 11:18
@incrypto32 incrypto32 force-pushed the krishna/file-link-resolver branch from cd2d014 to e153358 Compare June 10, 2025 11:29
@incrypto32 incrypto32 merged commit 29db3a5 into krishna/gnd-base Jun 10, 2025
6 checks passed
incrypto32 added a commit that referenced this pull request Jul 16, 2025
* graph: Add a new FIleLinkResolver

* graph: remove `/ipfs/` prefix when using file link resolver

* graph: Implement custom deserialise logic for Link to enable file link resolver

* tests: Add runner test that uses file link resolver

* graph: Conditionally disable deployment hash validation based on env var

* graph: use constant for "/ipfs/" prefix in `remove_prefix`

* graph: Simplify resolve_path by removing redundant path.is_absolute() check

* graph: Remove leftover println from file_resolver tests

* tests: Refactor runner tests extract test utils into recipe.rs

* tests: Add a test for file_link_resolver
incrypto32 added a commit that referenced this pull request Jul 18, 2025
* Refactor main function (#5980)

* node: Refactor main execution flow and introduce launcher module

* node/launcher: extract setup_configuration helper  from run

* node/launcher: extract setup_metrics helper  from run

* node/launcher: extract setup_store helper  from run

* node/launcher: extract build_blockchain_map helper  from run

* node/launcher: extract cleanup_ethereum_shallow_blocks helper  from run

* node/launcher: extract spawn_block_ingestor helper  from run

* node/launcher: extract deploy_subgraph_from_flag helper  from run

* node/launcher: extract spawn_contention_checker helper  from run

* node/launcher: extract build_graphql_server helper  from run

* node/launcher: extract build_subgraph_registrar helper  from run

* Implement a File Link Resolver (#5981)

* graph: Add a new FIleLinkResolver

* graph: remove `/ipfs/` prefix when using file link resolver

* graph: Implement custom deserialise logic for Link to enable file link resolver

* tests: Add runner test that uses file link resolver

* graph: Conditionally disable deployment hash validation based on env var

* graph: use constant for "/ipfs/" prefix in `remove_prefix`

* graph: Simplify resolve_path by removing redundant path.is_absolute() check

* graph: Remove leftover println from file_resolver tests

* tests: Refactor runner tests extract test utils into recipe.rs

* tests: Add a test for file_link_resolver

* Graph node dev mode (#5982)

* node: Create a new binary for graph node dev mode

* graph, store: Add unassign_subgraph method to SubgraphStore

* node: Add helpers for graph node dev for  subgraph management

* node: Add helper functions for watching files in dev mode

* node: Wire file watching in dev mode to redeploy subgraphs

* node: fix formatting

* gnd: Support multiple subgraphs, grafting, subgraph composition in dev mode (#6000)

* graph: Add clone_for_deployment to FileLinkResolver to create  FileLinkResolver with the right base dir for a subgraph

* graph: Add for_deployment to LinkResolverTrait

* core, graph: use for_deployment to get properly scoped resolver

* graph: Implement aliases for file link resolver

* node: Make gnd work with multiple subgraphs

* node: Support subgraph datasource in gnd

* node: correct the default value for manfiest

* core, node, graph: Ignore graft base in dev mode

* node: Allow providing a postgres url for gnd

* node: Do not use pgtemp in windows

* store: enable `vendored` feature for openssl crate

* chain/ethereum: Return error when ipc is used in non unix platform

* node: Refactor launcher

* node/dev : Better error message when database directory doesn't exist

* node: refactor watcher

* core, node, graph: Manipulate raw manifest instead of passing
ignore_graft_base

This reverts commit b5bbf93.

* node: Correct comments on `redeploy_all_subgraphs`

* node/gnd: Deploy all subgraphs first before wathcing files

* core, graph : Refactor LinkResolver trait

* Workflow to build the gnd binary (#6013)

* .github: Create a workflow for building gnd binaries

* .github: Codesign gnd binary for macOs

* .github: notarize gnd binary for macOs

* gnd: Integration tests (#6035)

* node/gnd: Make ports configurable

* node/gnd: Deploy all subgraphs on startup

* tests: Refactor subgraph datasources in TestCase

* tests: refactor Testcase method for source subgraphs

* tests: Add integration tests for gnd

* store: Use bundled pq-sys

* gnd: remove temp database directory on exit

* gnd: use pgtemp from graphprotocol org

* gnd: add alias for pgtemp db for windows

* gnd: use deep codesigning for macos binaries

* update workflow to add entitlements.plist
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants