8000 Is it possible to do partial parsing, fallback and wildcard ? · Issue #156 · lalrpop/lalrpop · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Is it possible to do partial parsing, fallback and wildcard ? #156

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
gwenn opened this issue Oct 11, 2016 · 10 comments
Open

Is it possible to do partial parsing, fallback and wildcard ? #156

gwenn opened this issue Oct 11, 2016 · 10 comments
Labels
wishlist

Comments

@gwenn
Copy link
Contributor
gwenn commented Oct 11, 2016

Hello,
I am trying to parse the SQLite3 grammar with LALRPOP here.
And some Lemon features do not seem to be avaible with LALRPOP.
Maybe I missed them while reading the documentation.

If they are not supported, it is not really important.
Many thanks for this great parser.

Partial parsing

SQLite parses one command at a time.
The user can choose to ignore the unparsed chunk (pzTail).
Or choose to parse the next command.

Lemon grammar

cmdx ::= cmd.           { sqlite3FinishCoding(pParse); }

For example, if the input is CREATE TABLE test (data); INSERT INTO test VALUES ('');.
SQLite parses the first statement CREATE TABLE test (data); and returns the unparsed chunk INSERT INTO test VALUES ('');.

Fallback

SQLite makes possible to use some keywords without double-quoting them where an identifier is expected.
For example, CREATE TABLE test (view TEXT); is a valid statement.
The view keyword becomes a simple identifer.

Lemon grammar

// An IDENTIFIER can be a generic identifier, or one of several
// keywords.  Any non-standard keyword can also be an identifier.
//
%token_class id  ID|INDEXED.

// The following directive causes tokens ABORT, AFTER, ASC, etc. to
// fallback to ID if they will not parse as their original value.
// This obviates the need for the "id" nonterminal.
//
%fallback ID
  ABORT ACTION AFTER ANALYZE ASC ATTACH BEFORE BEGIN BY CASCADE CAST COLUMNKW
  CONFLICT DATABASE DEFERRED DESC DETACH EACH END EXCLUSIVE EXPLAIN FAIL FOR
  IGNORE IMMEDIATE INITIALLY INSTEAD LIKE_KW MATCH NO PLAN
  QUERY KEY OF OFFSET PRAGMA RAISE RECURSIVE RELEASE REPLACE RESTRICT ROW
  ROLLBACK SAVEPOINT TEMP TRIGGER VACUUM VIEW VIRTUAL WITH WITHOUT
  EXCEPT INTERSECT UNION
  REINDEX RENAME CTIME_KW IF
  .

Wildcard

Lemon grammar

%wildcard ANY.

https://www.sqlite.org/src/doc/trunk/doc/lemon.html

The %wildcard directive is followed by a single token name and a period. This directive specifies that the identified token should match any input token.
When the generated parser has the choice of matching an input against the wildcard token and some other token, the other token is always used. The wildcard token is only matched if there are no other alternatives.
@brendanzab brendanzab mentioned this issue Oct 16, 2016
Merged
@brendanzab
Copy link
Contributor
brendanzab commented Oct 16, 2016 •
edited
Loading

This is also important for making grammars with nice editor integration. Eg. gluon needs it for its VS Code plugin:

@euclio euclio mentioned this issue Oct 18, 2016
Closed
@Marwes Marwes mentioned this issue Nov 8, 2016
Merged
7 tasks
@gwenn gwenn mentioned this issue Mar 25, 2017
Open
@ahmedcharles
Copy link
Contributor

I don't think these things are currently possible. I'm also not sure if they are desirable?

@phrohdoh
Copy link

@ahmedcharles Why would they not be desirable?

There are two, quite clear, requests for it above.

@ahmedcharles
Copy link
Contributor

I simply take the general position that the answer to most feature requests should be 'no, with a hint of maybe' rather than 'sure, why not'. There's a very insightful video here: https://www.youtube.com/watch?v=gXdS3IftP0Y

I'll paraphrase the core observation: If you are considering adding a feature and you make a choice that you later realize was wrong, one situation is significantly easier to fix than the other.

@phrohdoh
Copy link

Maintenance burden is unrelated to whether or not this feature is desirable.

@ahmedcharles
Copy link
Contributor

It depends. Is it desirable to implement or desirable to an arbitrary user? Communication is hard.

@gwenn
Copy link
Contributor Author
gwenn commented Jun 22, 2018

For fallback, there is #195.

@wycats
Copy link
Contributor
wycats commented Sep 24, 2018

I'll paraphrase the core observation: If you are considering adding a feature and you make a choice that you later realize was wrong, one situation is significantly easier to fix than the other.

That is only true if you ended up keeping the user for whom the question is relevant. It is nearly impossible to fix the problem caused by losing a user to a competing solution, so we ought not be this glib about rejected reasoned and use-case-derived feature requests.

At minimum, if you want to make the claim that a feature is undesirable, you ought to say why that is, and "saying no is better than saying yes" ought not be a sufficient justification.

@Dylan-DPC-zz
Copy link

That is only true if you ended up keeping the user for whom the question is relevant. It is nearly impossible to fix the problem caused by losing a user to a competing solution, so we ought not be this glib about rejected reasoned and use-case-derived feature requests.

This often happens in open source where someone needs some feature because he needs it for what he is using the crate for. But the feature doesn't align with the crate goals. I'm not saying this is the case here - but if often happens.

The thing is - we haven't had much progress over the last few months and it is difficult given each of our timeframes. It might be difficult to review an entire new feature or work on it from the perspective of a co-maintainer.

That said if @wycats or @gwenn or anyone else wants to give this a try and work out a PR it will be nice, though there are chances that it might be closed if it doesn't seem desirable. I will discuss this issue the next time i have a conversation with the other co-maintainers.

@wycats
Copy link
Contributor
wycats commented Sep 24, 2018

This often happens in open source where someone needs some feature because he needs it for what he is using the crate for. But the feature doesn't align with the crate goals. I'm not saying this is the case here - but if often happens.

Of course, and in such a case it's quite reasonable to say no. It is also quite reasonable to say "I have no time to look into this yet".

I find it unreasonable to say "I'm just going to assume this feature might be undesirable unless you work very hard to prove otherwise".

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
< 4358 span class="css-truncate js-issue-assignees"> No one assigned
Labels
wishlist
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
6 participants
@wycats @gwenn @brendanzab @ahmedcharles @phrohdoh @Dylan-DPC-zz
0