[JsonPath] Add the component #59655

alexandre-daubois · 2025-01-30T13:41:45Z

Q	A
Branch?	7.3
Bug fix?	no
New feature?	yes
Deprecations?	no
Issues	Fix #57280
License	MIT

JsonPath component

Today I'm presenting the JsonPath component. Thanks to this component, it will be possible to query JSON strings using the JSON Path syntax, as described in the recent RFC 9535. This RFC was released in February 2024 and is published here: https://datatracker.ietf.org/doc/html/rfc9535.

Here's a preview of what's possible:

<?php

require 'vendor/autoload.php';

use Symfony\Component\JsonPath\JsonPath;
use Symfony\Component\JsonPath\JsonCrawler;

$json = <<<'JSON'
{"store": {"book": [
    {"category": "reference", "author": "Nigel Rees", "title": "Sayings", "price": 8.95},
    {"category": "fiction", "author": "Evelyn Waugh", "title": "Sword", "price": 12.99}
]}}
JSON;

$crawler = new JsonCrawler($json);

// Basic property access
$result = $crawler->find('$.store.book[0].title');
// Array slice
$result = $crawler->find('$.store.book[0:2]');
// Reverse array slice
$result = $crawler->find('$.store.book[::-1]');
// Filter expression
$result = $crawler->find('$.store.book[?(@.price < 10)]');
// Recursive descent
$result = $crawler->find('$..author');

// Call to "length()"
$result = $crawler->find('$.store.book[?length(@.author) > 11]');

// Call to "match()"
$result = $crawler->find('$.store.book[?match(@.author, "[A-Z].*el.+")]');

// use a builder to create your path
$path = new JsonPath();
$path = $path->key('book')
    ->index(0)
    ->key('author');

$result = $crawler->find($path);

As stated in RFC 9535, this component embeds a few read-to-use functions:

length
count
value
search
match

Integration of JsonStreamer for performance

Thanks to the powerfulness of JsonStreamer's Splitter, we're able to guess which part of the JSON needs and doesn't need to be decoded. Querying a whole node means we can only json_decode() the substring containing the node. The same goes when encountering array keys. We extract the relevant substring thanks to Splitter that provides an API for that. It brings support for JSON-as-resource evaluation.

We stop when we encounter an operation that implies filtering as we need to decode the whole node to filter children.

What about other packages?

A few alternatives exist, however they were not updated in the last few years. If they are up-to-date, they do not seem to be following RFC 9535, but rather a partial implementation of https://goessner.net/articles/JsonPath/.

Why not include it in DomCrawler or PropertyAccess?

That was my first thought, however DomCrawler and JsonPath actually share absolutely no logic and their purpose is really different. As they have not much in common, that would be no ideal to tie them up.

PropertyAccess could be another possibility, here's where I explained why I think this would not be a correct fit: #59655 (comment).

Does it need external dependencies?

No! This component is written in vanilla PHP and doesn't require any third-party package to work.

How will it be leveraged in Symfony/PHP?

So many possibilities 😉 The first I can think of is leveraging this component in integration tests.

Indeed, this would allow to easily validate and write new assert methods when writing integration tests with Symfony. Validating JSON returned by an API thanks to this notation would be way easier (and more readable). In short, HttpClient, BrowserKit and WebTestCase could beneficiate from this (especially BrowserKitAssertionsTrait where asserts on Json could be added!).

Symfony would not be the only beneficiary: we can easily imagine that libraries like Behat (or PHPUnit, why not) could use this package to implement asserts on JSON.

Apart from testing frameworks, this package can also be used to quickly extract precise data from a JSON. So, possibilities here are also endless.

Possible evolution

Leverage ExpressionLanguage to evaluate filters and allow user to inject its own filter functions (custom functions are allowed per the RFC)
Improve the JsonPath builder
XPath may be converted to JSON Path fairly easily, this could be something to investigate on

src/Symfony/Component/JsonCrawler/Exception/InvalidJsonException.php

mvhirsch · 2025-01-30T14:00:11Z

Symfony would not be the only beneficiary: we can easily imagine that libraries like Behat (or PHPUnit, why not) could use this package to implement asserts on JSON.

Not to forget zenstruck/browser which has opt-in support for jmespath. Having native support for JSONPath might be awesome. WDYT @kbond?

hantsy · 2025-01-31T10:57:25Z

As stated in RFC 9535, this component embeds a few read-to-use functions:

Add the raw jsonPath function to evaluate the JsonPath expression directly.

Kocal

Cool!

src/Symfony/Component/JsonCrawler/JsonCrawler.php

kbond · 2025-01-31T13:15:03Z

Not to forget zenstruck/browser which has opt-in support for jmespath. Having native support for JSONPath might be awesome. WDYT @kbond?

Absolutely! That's the first thing I thought of.

dunglas · 2025-01-31T15:07:54Z

Why "JsonCrawler" and not "JsonPath"?

src/Symfony/Component/JsonCrawler/JsonCrawler.php

src/Symfony/Component/JsonCrawler/Tokenizer/JsonPathTokenizer.php

src/Symfony/Component/JsonCrawler/JsonCrawler.php

alexandre-daubois · 2025-01-31T15:33:49Z

Why "JsonCrawler" and not "JsonPath"?

Had the same thought earlier today, happy to see I'm not the only one! Let's see if anybody's against this naming. Otherwise, let's call it JsonPath, makes more sense to me as well.

GromNaN

I also think that JsonPath would be more appropriate as this is a well-known name of the spec that is implemented.

But maybe it should be part of the PropertyAccess? component. Why restrict this rich syntax to JSON documents when you can browse a tree of PHP objects in the same way. It would give more power to the PropertyAccess component.

src/Symfony/Component/JsonCrawler/JsonCrawler.php

alexandre-daubois · 2025-02-01T11:59:00Z

@GromNaN I think that's a good question. Here's why I'm not 100% sure that would be a fit.

Why restrict this rich syntax to JSON documents when you can browse a tree of PHP objects in the same way

Given the complexity of PropertyAccess (the logic to know if a property is writable/readable, get the value, read it, etc. is far from being trivial), I'm afraid it would bring a lot more complexity to JsonPath. The few times I fixed something in PropertyAccess was a whole challenge by itself just to understand all supported cases. JsonPath only deals with always-readable scalar values in arrays, which makes things way more simple. However, what would be possible is to create something like a converter in a second time, to convert a JsonPath to a PropertyPath. PropertyAccess could then have an opt-in possibility to be used together with JsonPath.

It would give more power to the PropertyAccess component

Definitely, but wouldn't it be too much power actually? PropertyAccess is, again, very complex and supports a lot of cases. But let's put implementation complexity aside. In my opinion, it would be very confusing for developers to have, in the same component, two different syntaxes that look alike, but doesn't have the same possibilities at all. For newcomers, it could be really puzzling.

Finally, if external packages (like testing frameworks and the ones I mentioned in the description) would like to support JsonPath, I'm not sure we should force to install the whole PropertyAccess component with it.

But I still think that having an opt-in dependency to JsonPath in PropertyAccess, to convert from one syntax to another, is something that may be investigated!

alexandre-daubois · 2025-02-01T14:38:40Z

Thank you @GromNaN and @stof for your review, I addressed all comments and renamed the component JsonPath.

src/Symfony/Component/JsonCrawler/README.md

src/Symfony/Component/JsonCrawler/Exception/JsonCrawlerException.php

src/Symfony/Component/JsonCrawler/Exception/InvalidJsonPathException.php

src/Symfony/Component/JsonCrawler/Exception/InvalidInputJsonException.php

src/Symfony/Component/JsonPath/Exception/InvalidInputJsonException.php

src/Symfony/Component/JsonPath/Exception/InvalidJsonPathException.php

src/Symfony/Component/JsonPath/Exception/JsonCrawlerException.php

src/Symfony/Component/JsonPath/README.md

src/Symfony/Component/JsonPath/composer.json

fabpot · 2025-03-03T07:10:44Z

@alexandre-daubois You can rebase this one to use the new JsonStreamer component :)

alexandre-daubois · 2025-03-03T07:55:31Z

Updated with JsonStreamer, thank you!

src/Symfony/Component/JsonPath/JsonCrawlerInterface.php

src/Symfony/Component/JsonPath/Exception/InvalidInputJsonException.php

alexandre-daubois · 2025-03-19T11:01:16Z

Rebased on 7.3.

Status: Needs Review

src/Symfony/Component/JsonPath/README.md

composer.json

src/Symfony/Component/JsonPath/JsonCrawlerInterface.php

nicolas-grekas

Thinking loud about the scope and the name of the component: would it be the place to support navigating hypermedia APIs?

alexandre-daubois · 2025-03-28T12:45:40Z

It deserves further investigation about the scope indeed, but we could imagine a JsonNavigator component (or something like that) that combines both JSON path manipulation and hypermedia JSON+LD APIs navigation. At first glance, it seems reasonable to me to have both under one component as JsonPath actually "navigates" through JSON trees.

fabpot · 2025-03-28T12:58:22Z

Thank you @alexandre-daubois.

src/Symfony/Component/JsonPath/JsonCrawler.php

…(alexandre-daubois) This PR was merged into the 7.3 branch. Discussion ---------- [JsonPath] Fix error message when evaluating a resource | Q | A | ------------- | --- | Branch? | 7.3 | Bug fix? | no | New feature? | no | Deprecations? | no | Issues | - | License | MIT After #59655 (comment) Commits ------- e5cc3d4 [JsonPath] Fix error message when evaluating a resource

carsonbot added Status: Needs Review Feature labels Jan 30, 2025

carsonbot added this to the 7.3 milestone Jan 30, 2025

alexandre-daubois force-pushed the json-crawler branch 3 times, most recently from e7e8192 to 32623b5 Compare January 30, 2025 13:52

OskarStark reviewed Jan 30, 2025

View reviewed changes

src/Symfony/Component/JsonCrawler/Exception/InvalidJsonException.php Outdated Show resolved Hide resolved

alexandre-daubois force-pushed the json-crawler branch from 32623b5 to 95e2a6a Compare January 30, 2025 13:57

alexandre-daubois force-pushed the json-crawler branch 2 times, most recently from 19f1073 to 5669a25 Compare January 30, 2025 14:01

Kocal reviewed Jan 31, 2025

View reviewed changes

src/Symfony/Component/JsonCrawler/JsonCrawler.php Outdated Show resolved Hide resolved

alexandre-daubois force-pushed the json-crawler branch from 5669a25 to b32574b Compare January 31, 2025 13:24

stof reviewed Jan 31, 2025

View reviewed changes

GromNaN reviewed Jan 31, 2025

View reviewed changes

src/Symfony/Component/JsonCrawler/JsonCrawler.php Outdated Show resolved Hide resolved

src/Symfony/Component/JsonCrawler/JsonCrawler.php Outdated Show resolved Hide resolved

alexandre-daubois force-pushed the json-crawler branch 2 times, most recently from 1d6f001 to ce60d57 Compare February 1, 2025 14:36

alexandre-daubois changed the title ~~[JsonCrawler] Add the component~~ [JsonPath] Add the component Feb 1, 2025