Closed
Description
This code panics on the marked line. It seems like IsValid() incorrectly returns true when it should return false since there are no more tokens available.
p := tokenizer.New()
stream := p.ParseString("")
defer stream.Close()
for stream.IsValid() {
token := stream.CurrentToken()
if token.Is(tokenizer.TokenInteger) { // <--- panic
fmt.Println(token.ValueInt())
}
stream.GoNext()
}
I'm using 1.3.0 and the panic stack trace tells me the problem is here:
github.com/bzick/tokenizer.(*Token).Is(...)
/xxx/go/pkg/mod/github.com/bzick/tokenizer@v1.3.0/token.go:220