Skip to content

[Breaking Change] Allow styling nested tokens with classes in styles #305

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 22 commits into from
Oct 4, 2020

Conversation

karlhorky
Copy link
Contributor

@karlhorky karlhorky commented Aug 24, 2020

First of all, thanks for this project, very cool!

I'm just opening an early version of a potential solution for #204 to allow styling things like .rule (to override .atrule) and .interpolation-punctuation (to override .punctuation).

This allows to style the interpolation-punctuation differently than the normal punctuation like this:

Screen Shot 2020-08-24 at 15 09 51

import { Prism as SyntaxHighlighter } from 'react-syntax-highlighter';

const style = {
  punctuation: {
    color: '#ff0000',
  },
  'punctuation.interpolation-punctuation': {
    color: '#00ff00',
  },
};

const Component = () => {
  const codeString = 'const a = \`\${b}\`;';
  return (
    <SyntaxHighlighter language="javascript" style={style}>
      {codeString}
    </SyntaxHighlighter>
  );
};

TODO

  • Recognize selectors with multiple class names and apply styles
  • Change script to build stylesheets (atrule.url instead of atrule .token.url)
  • Address test failures

Why is this a breaking change?

TLDR; This is a breaking change because it changes the way theme keys are authored (the new style is eg. atrule.url instead of atrule .token.url).

The old style (atrule .token.url) wasn't working before anyway (see #204), so this is technically not breaking the behavior of themes:

  • they didn't work before
  • without changes, they will also not work with the new version

However, the benefit is, if the themes are changed to the new format, colors that never worked before will start working in the new version. The change required is minimal:

-"atrule .token.url": {
+"atrule.url": {
    "color": "#9cdcfe"
},
-"atrule .token.url .token.function": {
+"atrule.url.function": {
    "color": "#dcdcaa"
},
-"atrule .token.url .token.punctuation": {
+"atrule.url.punctuation": {
    "color": "#d4d4d4"
},

Why was it like that? The refractor stylesheets script did a simple conversion on this CSS without much processing:

.token.atrule .token.url {
	color: #9cdcfe;
}

.token.atrule .token.url .token.function {
	color: #dcdcaa;
}

.token.atrule .token.url .token.punctuation {
	color: #d4d4d4;
}

It was transformed into this JS (note the extra .token bits):

"atrule .token.url": {
    "color": "#9cdcfe"
},
"atrule .token.url .token.function": {
    "color": "#dcdcaa"
},
"atrule .token.url .token.punctuation": {
    "color": "#d4d4d4"
},

The script has now been updated in this PR, so the new correct format will be generated:

"atrule.url": {
    "color": "#9cdcfe"
},
"atrule.url.function": {
    "color": "#dcdcaa"
},
"atrule.url.punctuation": {
    "color": "#d4d4d4"
},

Why the new format?

The flat structure of the DOM in react-syntax-highlighter (see #305 (comment)) means it makes more sense to author each key similar to a CSS selector with multiple chained classes (eg. atrule.url). This is different from the current format, which was a simple conversion of the Prism theme selectors without processing.

It also makes it more efficient and less ambiguous to match the theme keys to the node className: https://github.com/karlhorky/react-syntax-highlighter/blob/5cf65a0bf828c0f02cee033cbb895670ac1a4e88/src/create-element.js#L5-L39

Original issue

See problem reproduction here: #204 (comment)

Ref #204

@simmerer
Copy link
Collaborator

Glad to see you've tackled this... I had a similar nested-token-styling issue in a different project that relies on RSH, and hadn't yet attempted a solution. I'll take a look as soon as I can.

@karlhorky
Copy link
Contributor Author

karlhorky commented Aug 28, 2020

Great, thanks!

Since I've worked a bunch on PrismJS/prism-themes#103 recently, I guess there is a weakness in this approach:

It doesn't cover the case of features requiring descendant combinator CSS (the space), such as .token.imports .token.maybe-class-name or .token.atrule .token.rule:

PrismJS/prism#2533

Edit: this is because these nested elements do not yet exist in react-syntax-highlighter (see #305 (comment))

@karlhorky
Copy link
Contributor Author

karlhorky commented Aug 28, 2020

Ah I guess there's a DOM element missing on the react-syntax-highlighter side for this:

prism (from https://prismjs.com/test.html#language=css)

Screen Shot 2020-08-28 at 17 20 48

react-syntax-highlighter (from https://react-syntax-highlighter.github.io/react-syntax-highlighter/demo/prism.html)

Screen Shot 2020-08-28 at 17 20 26

refractor

refractor does already return this structure from refractor.highlight (demo: https://runkit.com/embed/mfigg7b3rcb5):

Screen Shot 2020-08-28 at 17 46 12

Code

@media (max-width: 800px) {
width: 400px;
}

@karlhorky
Copy link
Contributor Author

@simmerer if I can ask, is there a requirement by react-syntax-highlighter to NOT include this extra DOM element for whatever reason?

Because otherwise, I could also take a look at adding in these extra containers...

@simmerer
Copy link
Collaborator

simmerer commented Aug 28, 2020

@karlhorky Not that I know of. @conorhastings is the original creator of this package (I just stepped in to assist), so he might have an opinion.

In my understanding, Prism (or Highlight) themes should work the same in react-syntax-highlighter as if you were using Prism (or Highlight) directly, which means the DOM should provide all of the expected structure. If refractor provides those elements, we probably should as well.

We'd be super grateful for your help on this!

@karlhorky
Copy link
Contributor Author

Ok sounds good. I'll take a look at this over the next days, see if I can figure out why that DOM structure is not being reproduced, and try my hand at fixing it.

@karlhorky
Copy link
Contributor Author

karlhorky commented Aug 29, 2020

It looks like the flattening behavior comes from a flattenCodeTree function written in 2017 by @conorhastings - #64

It was originally introduced to avoid some undesirable wrapping behavior (see #63).

I guess the problems / difficulties are caused because of code like this:

@media (



max-width: 800px) {
width: 400px;
}

This is the structure that comes out of this (see example: https://runkit.com/karlhorky/5f4a5c661cb06e001acdff31):

Screen Shot 2020-08-29 at 16 07 30

This makes it potentially complicated to work around these issues (see wooorm/refractor#3). That's why #46 is somewhat complicated.

Maybe we can work around the wrapping problems differently.

@karlhorky
Copy link
Contributor Author

Does the wrapLines option actually work? Can't find an example where it actually wraps to the container width properly.

@karlhorky
Copy link
Contributor Author

karlhorky commented Aug 29, 2020

Oh I guess wrapLines does not mean what I (and others) think it does.

Looking at the original issue that motivated this change #44, it's about wrapping each line in a container element, not wrapping when the code size exceeds the width.

Maybe the option name should be changed to something less ambiguous...

@karlhorky
Copy link
Contributor Author

Ok @simmerer I gave it another shot, this time styling all permutations and array subsets (power sets) of the classnames on a node.

It avoids having to rework the DOM structure (which is that way for a reason), and should be pretty performant (usually adds up to only a few milliseconds to style a whole document).

Let me know what you think, and if it's acceptable, we can try to land this (I suppose we'll need tests? or to adapt existing ones?)

@karlhorky
Copy link
Contributor Author

karlhorky commented Aug 30, 2020

Oh actually, wait - why am I trying to do this in an algorithmic way (such as my solution or this Stack Overflow answer) when it can just be declarative?

I don't think the number of class names will be greater than 3. Even with 4, adding all combinations would still be faster.

This is way better and more performant:

- function recursivePermute(arr, permutation = []) {
-   return arr.length === 0
-     ? [permutation]
-     : arr.flatMap((item, i, arr) =>
-         recursivePermute(
-           arr.filter((_, j) => j !== i),
-           [...permutation, item]
-         )
-       );
- }
- 
- const permutations = {};
- function permute(arr) {
-   const key = arr.join(".");
-   if (!permutations[key]) {
-     permutations[key] = recursivePermute(arr);
-   }
-   return permutations[key];
- }
- 
- function getSubArrays(arr) {
-   if (arr.length === 1) return [arr];
-   else {
-     const subarr = getSubArrays(arr.slice(1));
-     return subarr.concat(
-       subarr.map((el) => el.concat(arr[0])),
-       [[arr[0]]]
-     );
-   }
- }
+ // Get all possible permutations of all power sets
+ //
+ // Super simple, non-algorithmic solution since the
+ // number of class names will not be greater than 3
+ function powerSetPermutations(arr) {
+   if (arr.length === 0) return [];
+   if (arr.length === 1) return arr;
+   if (arr.length === 2) {
+     return [
+       arr[0],
+       arr[1],
+       `${arr[0]}.${arr[1]}`,
+       `${arr[1]}.${arr[0]}`,
+     ]
+   }
+   if (arr.length >= 3) {
+     // Currently does not support more than 3 class names
+     return [
+       arr[0],
+       arr[1],
+       arr[2],
+       `${arr[0]}.${arr[1]}`,
+       `${arr[0]}.${arr[2]}`,
+       `${arr[1]}.${arr[0]}`,
+       `${arr[1]}.${arr[2]}`,
+       `${arr[2]}.${arr[0]}`,
+       `${arr[2]}.${arr[1]}`,
+       `${arr[0]}.${arr[1]}.${arr[2]}`,
+       `${arr[0]}.${arr[2]}.${arr[1]}`,
+       `${arr[1]}.${arr[0]}.${arr[2]}`,
+       `${arr[1]}.${arr[2]}.${arr[0]}`,
+       `${arr[2]}.${arr[0]}.${arr[1]}`,
+       `${arr[2]}.${arr[1]}.${arr[0]}`,
+     ]
+   }
+ }

const classNameCombinations = {};
function getClassNameCombinations(classNames) {
  if (classNames.length === 0) return [];
  if (classNames.length === 1) return classNames;
  const key = classNames.join(".");
  if (!classNameCombinations[key]) {
-     classNameCombinations[key] = getSubArrays(classNames)
-       .flatMap((classNames) => classNames.length > 1 ? permute(classNames) : [classNames])
-       // Less specific (fewer class names) to more specific (more class names)
-       .sort((a, b) => (a.length < b.length ? -1 : 1))
-       .map((arr) => arr.join("."));
+     classNameCombinations[key] = powerSetPermutations(classNames);
  }
  return classNameCombinations[key];
}

@karlhorky
Copy link
Contributor Author

Ok, this is looking much cleaner now.

One breaking change that this will require is that class names in the styles no longer have the spaces that they currently have. I have noted this above in the description.

@karlhorky karlhorky changed the title Allow styling nested tokens with classes in styles [Breaking Change] Allow styling nested tokens with classes in styles Aug 30, 2020
@karlhorky
Copy link
Contributor Author

karlhorky commented Aug 30, 2020

Alright I've also fixed the build script now, seems to be working. I've marked this as a breaking change (mentioned in the description why).

@karlhorky
Copy link
Contributor Author

karlhorky commented Aug 30, 2020

Ok, I've also ran the tests and found some other incompatibilities that I've now fixed.

The only 2 tests that are failing also fail on the master branch:

Screen Shot 2020-08-30 at 16 00 11

So this PR should be ready to go!

cc @simmerer

@simmerer
Copy link
Collaborator

Great work here @karlhorky. I should be able to take a look tomorrow... thanks for your patience.

In regards to wrapLines, I've got a PR here which introduces a wrapLongLines prop that does what everyone thought wrapLines did. Would love your thoughts on that: #309

@karlhorky
Copy link
Contributor Author

Cool! Wrote a few comments. Not sure about the overall approach and if there aren't any edge cases with using pre-wrap, but seems like it could be legit!

@karlhorky
Copy link
Contributor Author

@simmerer I guess you haven't had a chance to take a look at this one yet, right? Any estimate on when you'll be able to take a look?

@simmerer simmerer self-requested a review October 2, 2020 04:09
Copy link
Collaborator

@simmerer simmerer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for taking so long to look this over! Generally looks great.

My one minor request is to trade out .flat() for a spread/concat operation, since this project still builds against node 10 for now.

@simmerer simmerer merged commit 05b4b8f into react-syntax-highlighter:master Oct 4, 2020
@karlhorky karlhorky deleted the patch-1 branch October 4, 2020 09:55
@karlhorky
Copy link
Contributor Author

Thanks for the merge @simmerer!

Super exciting that this is out now in 15.0.1 (sorry for missing the fixes on the demo themes)!

I'll give this a shot soon!

@simmerer
Copy link
Collaborator

simmerer commented Oct 4, 2020

No worries. npm publish (which runs npm run build) is still frequently the point at which I realize I haven't run npm run build to update the demos. :)

@karlhorky
Copy link
Contributor Author

karlhorky commented Oct 5, 2020

Ah, @simmerer would it be possible to update to the latest prism-themes@1.4.1 version too? (similar to your commit 7e26d31 ?)

I did a pull request to improve the accuracy of the VS Code Dark theme: PrismJS/prism-themes#103 (comment)

If you want, I can also open a new issue for this.

@simmerer
Copy link
Collaborator

simmerer commented Oct 5, 2020

I'll have a PR up shortly!

@simmerer
Copy link
Collaborator

simmerer commented Oct 5, 2020

🎉 Released in 15.1.0: https://github.com/react-syntax-highlighter/react-syntax-highlighter/releases/tag/v15.1.0

@karlhorky
Copy link
Contributor Author

Awesome, thanks for that!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants