8000 feat(developer): next generation KMN compiler - parser 🤔 by markcsinclair · Pull Request #13492 · keymanapp/keyman · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

feat(developer): next generation KMN compiler - parser 🤔 #13492

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 250 commits into
base: feat/developer/13349-kmc-kmn-next-generation-lexer
Choose a base branch
from

Conversation

markcsinclair
Copy link
Contributor
@markcsinclair markcsinclair commented Mar 13, 2025

A recursive-descent parser with AST generation for the next generation KMN compiler, based on a modified port of Fowler's parser (see Domain Specific Languages by Martin Fowler and Rebecca Parsons, Ch.21, 24)

see Next Generation KMN Compiler #13349

The parser will initially be developed for positive path only i.e. no error messages, and with no backward compatability

@keymanapp-test-bot skip

To do list (following first full parse of khmer_angkor.kmn):

  • Use correct nodes in AST for layer(), platform() and baselayout()`
  • Refactore system and variable store rules into separate file
  • Reduce rule object construction/destruction overhead using builder pattern
  • Include full range of current language elements
    • CALL
    • CAPSALWAYSOFF
    • CAPSONONLY
    • DEADKEY
    • NOTANY
    • NUL (on LHS)
    • RESET
    • RETURN
    • SAVE
    • SET
    • SHIFTFREESCAPS
  • Include named constants and Hangeul syllables (needs lexer support)
  • Check whitespace/continuation in rules for consistency and coverage
  • Check LINE nodes moved to correct places in AST
  • Ensure virtual keys are included only in keystroke part of rules
  • Include virtual character keys
  • Check keywords are only recognised where they are allowed
  • Include permitted keywords in variable store and group names
  • Tests for tree-construction.ts
  • Check test coverage

@keymanapp-test-bot
Copy link
keymanapp-test-bot bot commented Mar 13, 2025

User Test Results

Test specification and instructions

User tests are not required

@keymanapp-test-bot keymanapp-test-bot bot added this to the B18S3 milestone Mar 13, 2025
@markcsinclair markcsinclair marked this pull request as draft March 13, 2025 14:33
@markcsinclair markcsinclair changed the title feat(developer): initial version of TokenBuffer feat(developer): next generation KMN compiler - parser Mar 13, 2025
srl295
srl295 previously approved these changes Mar 13, 2025
Copy link
Member
@srl295 srl295 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some suggestions on expanding the ternary consitionals and on the Rule hierarchy but no objection to what's here!

@srl295 srl295 self-requested a review March 13, 2025 17:56
@mcdurdin
Copy link
Member

@srl295 process note: please don't approve PRs while still in draft.

@mcdurdin mcdurdin dismissed srl295’s stale review March 13, 2025 22:26

PR was in draft

@srl295
Copy link
Member
srl295 commented Mar 13, 2025

@srl295 process note: please don't approve PRs while still in draft.

accidental- thought i dismissed it.

@darcywong00 darcywong00 modified the milestones: B18S3, B18S4 Mar 14, 2025
…eration-lexer' into feat/developer/13349-kmc-kmn-next-generation-parser
…, but 2/10 of the ranges failing due to timeouts
…eration-lexer' into feat/developer/13349-kmc-kmn-next-generation-parser
…eration-lexer' into feat/developer/13349-kmc-kmn-next-generation-parser
…eration-lexer' into feat/developer/13349-kmc-kmn-next-generation-parser
@keyman-server keyman-server modified the milestones: A19S4, A19S5 Jun 8, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: No status
Development

Successfully merging this pull request may close these issues.

5 participants
0