Add \tl_retokenize:n #1761
Add \tl_retokenize:n #1761
Conversation
|
Thoughts on the name welcome: I'm aiming for different-from-rescan as this is 'dynamic', but wasn't sure that was the best term. (Context: I need this for |
|
|
l3kernel/l3tl.dtx
Outdated
| \group_begin: | ||
| \int_set:Nn \tex_newlinechar:D { 13 } | ||
| \exp_after:wN \group_end: | ||
| \tex_scantokens:D \exp_after:wN { \tl_to_str:n {#1} } |
There was a problem hiding this comment.
What happened to the \scantextokens idea? Should we try to add something that will (most likely) remove the trailing space? If not, the fact that if this is used in horizontal mode it'll have a trailing space in output should be documented!
There was a problem hiding this comment.
I couldn't find a way to do the same as \scantextokens reliably in pdfTeX - ideas welcome
There was a problem hiding this comment.
Exactly the same behaviour won't be possible (the grabbing additional arguments for instance). The space-suppression is possible, but only sort of unreliable. Provided the argument doesn't contain a NUL char (or any other arbitrary char), and doesn't change the category code of that character (the latter will be impossible to check, so we have to hope for the best here), you can put \char_set_catcode_comment:n { 0 } in the setup code and put a catcode 12 NUL char at the end of the \tex_scantokens:D argument.
There was a problem hiding this comment.
Or persuade someone (not me, I've given up on grasping WEB) to back port it...
There was a problem hiding this comment.
I've looked at a back-port - quite a bit of work and not a high priority.
|
Is The LuaMetaTeX version of
but running the following ConTeXt example shows that \starttext
\show\scantokens
\stoptext |
|
I've updated to allow for |
|
I guess I could ask Hans about |
|
|
| % Hans that LuaMeta\TeX{} will retain this primitive, so we should be | ||
| % OK for all engines. | ||
| % \begin{macrocode} | ||
| \cs_gset_protected:Npn \tl_rescan_dynamic:n #1 |
There was a problem hiding this comment.
Should be new then.
| \cs_gset_protected:Npn \tl_rescan_dynamic:n #1 | |
| \cs_set_protected:Npn \tl_rescan_dynamic:n #1 |
There was a problem hiding this comment.
This is deliberate - as we are wanting a global def (yes, i know we should be at top-level but if you look at new it works globally).
There was a problem hiding this comment.
This is deliberate - as we are wanting a global def (yes, i know we should be at top-level but if you look at new it works globally).
|
As |
| % régime. In contrast to \cs{tl_rescan:nn}, this function retokenizes | ||
| % the input \enquote{dynamically}: category code changes which occur | ||
| % whilst processing the retokenized material \emph{will} apply to later | ||
| % content. The \emph{tokens} are always treated as ending with a space. |
There was a problem hiding this comment.
Is it clear what is meant by this sentence?
| @@ -2213,6 +2229,23 @@ | |||
| % \end{macro} | |||
| % \end{macro} | |||
| % | |||
| % \begin{macro}{\tl_rescan_dynamic:n, \tl_rescan_dynamic:V} | |||
| % A thin wrapper from \tn{scantokens}. At present, we are told by | |||
No, the whole issue with |
No description provided.