Open
Description
First of all. Thanks for making this excellent library. I find it much more approachable, and yet more customizable and elegant than the other libraries on the market. It's a huge improvement over the PLY library. I hope to see this project grow in the future.
I'm having a small issue where I run into an IndexError when accessing an item within an optional which is within a repeat.
Please see the following simplified example.
import sly
class Lexer(sly.Lexer):
tokens = {A, B}
literals = {"[", "]", ","}
A = "a"
B = "b"
class Parser(sly.Parser):
tokens = Lexer.tokens
@_("'[' [ expr ] { ',' [ expr ] } ']'")
def array(self, p):
items = [p.expr0, *p.expr1]
# Each None accounts for an Elision
items = [item or ("elision", None) for item in items]
return "array", items
@_("A", "B")
def expr(self, p):
return "expr", p[0]
lexer = Lexer()
parser = Parser()
code = "[a,,a,b,,]"
tokens = list(lexer.tokenize(code)) # pulling full tokens, just so we can look at them.
print("\n".join(map(str, tokens)))
tree = parser.parse(iter(tokens))
print(repr(tree))Token(type='[', value='[', lineno=1, index=0)
Token(type='A', value='a', lineno=1, index=1)
Token(type=',', value=',', lineno=1, index=2)
Token(type=',', value=',', lineno=1, index=3)
Token(type='A', value='a', lineno=1, index=4)
Token(type=',', value=',', lineno=1, index=5)
Token(type='B', value='b', lineno=1, index=6)
Token(type=',', value=',', lineno=1, index=7)
Token(type=',', value=',', lineno=1, index=8)
Token(type=']', value=']', lineno=1, index=9)
Traceback (most recent call last):
File "/home/jon/.config/JetBrains/PyCharmCE2020.3/scratches/scratch_36.py", line 40, in <module>
tree = parser.parse(iter(tokens))
File "/home/jon/.virtualenvs/test/lib/python3.8/site-packages/sly/yacc.py", line 2082, in parse
value = p.func(self, pslice)
File "/home/jon/.config/JetBrains/PyCharmCE2020.3/scratches/scratch_36.py", line 18, in array
items = [p.expr0, *p.expr1]
File "/home/jon/.virtualenvs/test/lib/python3.8/site-packages/sly/yacc.py", line 148, in __getattr__
return self._namemap[name](self._slice)
File "/home/jon/.virtualenvs/test/lib/python3.8/site-packages/sly/yacc.py", line 239, in <lambda>
namemap[k] = lambda s,i=index,n=n: ([x[n] for x in s[i].value]) if isinstance(s[i].value, list) else s[i].value[n]
File "/home/jon/.virtualenvs/test/lib/python3.8/site-packages/sly/yacc.py", line 239, in <listcomp>
namemap[k] = lambda s,i=index,n=n: ([x[n] for x in s[i].value]) if isinstance(s[i].value, list) else s[i].value[n]
IndexError: tuple index out of range
Process finished with exit code 1
With some debugging, I discovered this is occuring when I try to access p.expr1.
I can access it with indexing, but it seems like it is due to the second expr term being incorrectly wrapped in a tuple? Below is the workaround I implemented, but I'm curious on your take on this.
@_("'[' [ expr ] { ',' [ expr ] } ']'")
def array(self, p):
items = [p.expr0]
for item in p[2]:
comma, expr_tuple = item
# expr_tuple is a single element tuple for some reason
items.append(expr_tuple[0])
items = [item or ("elision", None) for item in items]
return "array", itemsMetadata
Metadata
Assignees
Labels
No labels