[WIP] Simplify core #3905

LeaVerou · 2025-04-26T20:28:14Z

No description provided.

netlify · 2025-04-26T20:28:29Z

✅ Deploy Preview for dev-prismjs-com ready!

Name	Link
🔨 Latest commit	`d110fa7`
🔍 Latest deploy log	https://app.netlify.com/projects/dev-prismjs-com/deploys/6871814853fbc700085e3a02
😎 Deploy Preview	https://deploy-preview-3905--dev-prismjs-com.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

github-actions · 2025-04-26T20:28:50Z

No JS Changes

Generated by 🚫 dangerJS against d110fa7

src/core/hooks.ts

DmitrySharabin · 2025-04-27T17:20:48Z

I can't comment on the corresponding line since it's not in the PR, but I can't understand why we need a list variable if we already have hooks. Moreover, we do const list = hooks. What am I missing?

add<Name extends string> (name: Name, callback: HookCallback<Name>): () => void {
	let hooks = this._all.get(name);
	if (hooks === undefined) {
		hooks = [];
		this._all.set(name, hooks);
	}
	const list = hooks;

	list.push(callback as never);

	return () => {
		const index = list.indexOf(callback as never);
		if (index !== -1) {
			list.splice(index, 1);
		}
	};
}

src/core/registry.ts

src/core.ts

src/global.ts

src/types.d.ts

src/core/prism-class.ts

Co-authored-by: Dmitry Sharabin <dmitrysharabin@gmail.com>

LeaVerou · 2025-04-27T18:52:10Z

src/core/prism-class.ts

+	 * See {@link highlight}
+	 */
+	highlight (text: string, language: string, options?: HighlightOptions): string {
+		return highlight.call(this, text, language, options);


I think this suggestion is on the wrong line

Co-authored-by: Dmitry Sharabin <dmitrysharabin@gmail.com>

src/core/prism-class.ts

LeaVerou · 2025-04-27T18:59:55Z

src/global.ts

-const globalPrism = (namespace[globalSymbol] ??= new Prism());
+declare global {
+	var Prism: Prism | undefined
+}


Nope, that's just what the code example I found for extending global was using 😛 But this suggestion is also on the wrong line.

Co-authored-by: Dmitry Sharabin <dmitrysharabin@gmail.com>

src/global.ts

@DmitrySharabin

cc @DmitrySharabin

src/core/tokenize/match.ts

* Types and type safety * Fix the build script * Remove unnecessary import * Import lost types + Prettier * Use `includes` instead of `indexOf` * Add type for better auto completion

DmitrySharabin · 2025-05-19T15:03:46Z

src/core/classes/component-registry.ts

+	 */
+	private loadingList: Promise<T>[] = [];
+
+	ready: Promise<T[]>;


Shouldn't ready be a getter to become reactive? We change dynamically loadingList (ready depends on it). However, we set the value of ready once when creating an instance of a registry and never update it. As a result, we have ready that is always fulfilled, even though there might be promises that are not fulfilled in the loadingList array.
Or should we update ready every time we update loadingList? Or is there another option I'm not seeing?

- If a grammar has a base language, it should always be resolved - Fix `getLanguage()`: `language` can already be a resolved grammar if it is taken from `this.languages`

- Don’t choke on patterns that are not instances of `RegExp` - In `resolve()` return `resolvedGrammar` and delete `$rest` after it is properly resolved

* Fix `test:components` * Partially adopt the new API * Fix `test:identifiers` * Use `languageRegistry` * Use cache * Fix core tests * Fix ESLint and TS errors + typo in `regex-coverage` * Fix ESLint and TS errors in `test:patterns`

DmitrySharabin · 2025-05-26T13:16:44Z

Things to implement before merging:

$self
Correctly resolve inside: 'language-id'

Something else?

…back to `any` (#3944)

LeaVerou · 2025-05-27T14:37:24Z

For $self, we need to figure out what that means when we have nested languages (e.g. highlighting JS in HTML). Probably the closest language.

inside: 'language-id' worked before, so that must be a regression.

DmitrySharabin · 2025-05-27T20:52:38Z

inside: 'language-id' worked before, so that must be a regression.

Yup, this is related to something we've already started to discuss.

We had an [issue](ony3000/prettier-plugin-merge#36) formatting complex files with many regexes, which is now fixed.

LeaVerou · 2025-05-28T15:20:02Z

inside: 'language-id' worked before, so that must be a regression.

Yup, this is related to something we've already started to discuss.

That does not address why it stopped working. This was even working in v1. And the code to make it work is still there I believe (in tokenize)

* [C] Fix transformation bug: `macro` should be inserted before `string` * [F#] Specify the base language * [extend] Deeply nested objects can be `undefined` * [C#] Specify the base language + remove unused imports * [PHP] Add missed tokens * [VB.NET] Fix language definition * [MD] Correctly capture the language code

…ot yet resolved (#3951)

- Add the missing inside token and fix the existing one - Fix the regex that is too greedy now: if we have more than one code block next to each other, all of them starting from the second one will be interpreted as a code block of the first one - Ignore spaces before the language code

* Vala * Firestore security rules * Xeora * Flow * Wiki * V * Velocity * glsl * gml * Go * gradle * groovy * haxe * kotlin * javadoc * squirrel * hlsl * idris * json5 * jsonp * less * n4js * objectivec * opencl * sqf * sparql * solidity * scala * reason * racket * qore * purescript * purebasic * protobuf * processing * plsql * phpdoc * mongodb * jolie * latte * parser * textile * qsharp * ruby * sass * scss * xquery * http * haml * rescript

…ditions/removals (#3961) Credit: https://github.com/mavoweb/mavo/blob/78efe2b9cadd09c1d131b8afd5fe2f38d5cfa8c7/src/mavo.js#L845 Partially addresses #3960

* Place plugin docs along with their source code (#3930) * [autoloader] Fix script name in the docs * [normalize-whitespace] Don't consider demo as a plugin * Generate `file-sizes.json` during the build (#3946) * Exclude plugins' `demo.js` files from type checking * Fix path to the core entry point in `components.json` * Move and rename themes (#3947) - Move themes to the `src` folder and remove the `prism-` prefix - Adjust the build process accordingly - Adjust themes' metadata: - Remove broken links - Add/update authors' full names - Update GitHub handles - Use Markdown-style links - In `components.json`, fix theme ids and add Lea Verou as the owner to the prism, dark, and funky themes * Update themes' docs (#3958) So that they follow the structure (we can parse and use on the website): ``` /** * <title> * * <description> * * @author <name> (<GitHub_username>) */ ``` Between the data chunks, there should be at least one empty line. * Move `combineCallbacks` to its own module (#3963) * Move the `Token` class under the `classes` folder * Move all utils for working with iterables to a separate module Remove unused utils * Simplify hooks * Move the `Prism` class, add Prism singleton * Simplify the `Prism` class Move to separate modules: - `highlightAll()` - `highlightElement()` - `highlight()` - `stringify()` * Correctly parse CLI arguments (#3972) Was fixed in #3949 (in the `simplify` branch). Move it here to make testing easier. * Simplify the `Prism` class even more - Move `tokenize()`, `_matchGrammar()` (and related code), and `resolve()` to separate modules. - Ditch the `tokenize` symbol in favor of the `$tokenize` special property. Since symbols are excluded from the `for…in` loop, and `$tokenize` is not a symbol, to preserve the previous behavior, we need to handle it in `_matchGrammar` explicitly. - Switch from `[tokenize]` to `$tokenize` in languages. --- Co-authored-by: Dmitry Sharabin <dmitrysharabin@gmail.com> * Ditch the `rest` symbol in favor of the `$rest` special property * Move language utils to `util/` and separate them out * Remove known plugins + type safety * Move `documentReady` to async utils * Make the global bundle of Prism v2 work the same way as Prism v1 does Now, the user can include Prism the way they used to (except v2 now uses the Autoloader plugin under the hood, so more languages can be highlighted right out of the box): ```html <link rel="stylesheet" href="themes/prism.css" /> <script src="prism.js"></script> ``` Besides, they have access to the Prism instance via `globalThis`. * Move Prism config to a separate file and fix the build options of the global bundle Co-Authored-By: Dmitry Sharabin <dmitrysharabin@gmail.com> * Fix some code after merging --------- Co-authored-by: Lea Verou <lea@verou.me>

[WIP] Simplify core

3beff85

LeaVerou changed the base branch from master to v2 April 26, 2025 20:28

LeaVerou added 9 commits April 26, 2025 16:45

highlightAll

46806b6

Just use properties on env, not a separate state property

e4f6634

Move singleton, eliminate state further

183817c

Finish removing HookState

d1097d8

Update prism-class.ts

e3e5db2

Delete hook-state.ts

c866261

Move more functions out of prism-class

8ce6c69

Merge branch 'v2' into simplify

1a2c60b

Move tokenize

52fca5d