latex2e: recorddeps to images rel to CWD (+ lots of fixes) [SF:patches:67] #599
Labels
Comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
{{ message }}
author: kirr79
created: 2010-03-01 17:02:04
assigned: None
SF_url: https://sourceforge.net/p/docutils/patches/67
Subject: [PATCH 1/8] Node.copy: don't forget to also setup .document/.source/.line in duplicate
Spot this by debugging Substitute transformation which does .deepcopy()
to substitute definitions.
This will be needed in the next patch, where I'll be teaching Inliner to
propagate .line for `inline' elements.
---
docutils/docutils/nodes.py | 23 +++++++++++++++++++----
1 files changed, 19 insertions(+), 4 deletions(-)
commenter: kirr79
posted: 2010-03-01 17:02:04
title: #67 latex2e: recorddeps to images rel to CWD (+ lots of fixes)
attachments:
commenter: kirr79
posted: 2010-03-01 17:05:12
title: #67 latex2e: recorddeps to images rel to CWD (+ lots of fixes)
attachments:
commenter: kirr79
posted: 2010-03-01 17:06:31
title: #67 latex2e: recorddeps to images rel to CWD (+ lots of fixes)
commenter: kirr79
posted: 2010-03-01 17:07:06
title: #67 latex2e: recorddeps to images rel to CWD (+ lots of fixes)
Subject: [PATCH 2/8] rst: teach Inliner to propagate .source & .line for inline elements
In particular this will be needed for substitution_reference in
combination with image substitution_definition, latex2e backend and
--record-dependencies. e.g. for
some/file.txt:
.. |up| image:: pics/up.png
text text text |up| ...
when generating dependencies to pics/up.png, writer will want to prepend
document source path to pics/up.png so that the prerequisite becomes
some/pics/up.png , and for this it needs .source attribute in image
node.
This changes one test in test_substitutions.py:
input:
.. |indirect1| replace:: |indirect2|
.. |indirect2| replace:: |Sub|
.. |sub| replace:: |indirect1|
Use |sub| and |indirect1| and |sub| again (and |sub| one more time).
-: expected
+: output
[...]
<system_message backrefs="id6" ids="id5" level="3" line="5" source="test data" type="ERROR">
Circular substitution definition referenced: "sub".
- <system_message backrefs="id8" ids="id7" level="3" source="test data" type="ERROR">
+ <system_message backrefs="id8" ids="id7" level="3" line="2" source="test data" type="ERROR">
? +++++++++
Look, previsouly the ERROR has not line=... at all!
But that line is incorrect - it should be 5. The problem here is with
Substitution transform incorrectly reporing it -- I'll fix it in the
next patch.
---
docutils/docutils/parsers/rst/states.py | 6 +++++-
.../test/test_transforms/test_substitutions.py | 2 +-
2 files changed, 6 insertions(+), 2 deletions(-)
commenter: kirr79
posted: 2010-03-01 17:07:32
title: #67 latex2e: recorddeps to images rel to CWD (+ lots of fixes)
Subject: [PATCH 3/8] Substitutions: unroll problematic refs to their origin in document when reporting errors
This is needed to correctly report erroneous line for substitution
references. For example for
.. |indirect1| replace:: |indirect2|
.. |indirect2| replace:: |Sub|
.. |sub| replace:: |indirect1|
Use |sub| and |indirect1| and |sub| again (and |sub| one more time).
we now correctly report line info for 3rd substitutrion reference to |sub|
being at line 5 instead of 2.
The problem with previous approach was that we were iteratively expanding refs
to defs, and then again defs to refs and so on, but during that process,
source/line info for nodes were changing accordingly, e.g. when first |sub|
gets replaced with |inderect1|, resulting node line is 3, then |indirect1| is
replaced with |indirect2| and node line is 1, and then |indirect2| is replaced
to |sub| (node line is 2) and boom, we hit the cycle (this is right), but the
line number (2) was wrong.
In order to avoid that, let's just store back refs to original substitution
reference, so that when there is a cycle, we can unroll it to the original
problematic node and report right info.
---
docutils/docutils/transforms/references.py | 8 +++++++-
.../test/test_transforms/test_substitutions.py | 2 +-
2 files changed, 8 insertions(+), 2 deletions(-)
commenter: kirr79
posted: 2010-03-01 17:08:27
title: #67 latex2e: recorddeps to images rel to CWD (+ lots of fixes)
Subject: [PATCH 4/8] Node.setup_child: teach it to perform recursive setup of document/source/line in grandchildren
neccessary e.g. for `node` in cases like:
1. create node
2. create container
3. container += node
4. document += container
because in such cases, node would be left without document/source/line
info attached.
And also, in parser/rst/states.py, we manually set .line in some places,
but .source is not initalized explicitely anywhere, and this for e.g.
---- 8< q.txt ----
.. include:: s.txt
---- 8< ----
---- 8< s.txt ----
reStructuredText Test Document
Footnotes
---------
.. [4] Here's an unreferenced footnote, with a reference to a
nonexistent footnote: [5]_.
text
---- 8< ----
results in a very weird behaviour -- it says there is a reference to
unknown target "5", on line 8, but file is q.txt insetead of s.txt!
So we have to ("option 1") either manually patch every place where .line
is setup to also setup .source (and .document), or better provide some
way for nodes to automatically do it. One way would be to use something
like document.new_node(node_type, *args), like we create document not
through document.__init__, but through utils.new_document(). And that
new_node() would also setup current node context (document/source/line/
etc...) so we'll avoid such problems at all.
Or, try to restore source/line context for grandchildren in
Node.setup_child().
Right now I prefer second change -- it's less intrusive, but in the
longer run, option 1 is more viable...
This is a really important change. Unfortunatelly it also makes a lot of
tests fail.
Some of the test failures are for the better - one example of this is
e.g. standalone_rst_html4css1.py where in ERROR we were reporting
*incorrect* file (functional/input/standalone_rst_html4css1.txt, line
363 instead of functional/input/data/standard.txt)!
However, where source changed from standalone_rst_html4css1.txt to
standard.txt, there is no line info now... I'll restore it later though.
Other failures look like this:
functional/input/data/standard.txt:
---- 8< ----
367 .. [CIT2002] Citations are text-labeled footnotes. They may be
368 rendered separately and differently from footnotes.
369
370 Here's a reference to the above, [CIT2002]_, and a [nonexistent]_
371 citation.
372
373 .. _Another Target:
- <system_message backrefs="id82" ids="id81" level="3" line="372" source="functional/input/data/standard.txt" type="ERROR">
+ <system_message backrefs="id82" ids="id81" level="3" line="371" source="functional/input/data/standard.txt" type="ERROR">
Unknown target name: "nonexistent".
---- 8< ----
---- 8< ----
input:
Duplicate external target_'s (different URIs):
.. _target: first
.. _target: second
-: expected
+: output
Duplicate external
target_
's (different URIs):
<system_message backrefs="id1" level="2" line="5" source="test data" type="WARNING">
Duplicate explicit target name: "target".
- <system_message backrefs="id3" ids="id2" level="3" line="1" source="test data" type="ERROR">
? ^
+ <system_message backrefs="id3" ids="id2" level="3" line="2" source="test data" type="ERROR">
? ^
---- 8< ----
or
---- 8< ----
input:
Unknown reference_.
-: expected
+: output
Unknown
reference_
.
- <system_message backrefs="id2" ids="id1" level="3" line="1" source="test data" type="ERROR">
? ---------
+ <system_message backrefs="id2" ids="id1" level="3" source="test data" type="ERROR">
Unknown target name: "reference".
---- 8< ----
etc...
But I still insist that modulo "option 1" this change is *right*. It's
just that there are lots of other bugs still present in docutils.
We now have 11 test failures. I'll address them all in the next patches.
---
docutils/docutils/nodes.py | 11 +++++++++++
.../expected/standalone_rst_html4css1.html | 2 +-
.../functional/expected/standalone_rst_latex.tex | 2 +-
.../expected/standalone_rst_pseudoxml.txt | 2 +-
4 files changed, 14 insertions(+), 3 deletions(-)
commenter: kirr79
posted: 2010-03-01 17:08:51
title: #67 latex2e: recorddeps to images rel to CWD (+ lots of fixes)
Subject: [PATCH 5/8] rst: teach Inliner to propagate .line for reference elements
This fixes 5 of 11 failing testcases, e.g. this
input:
Duplicate external target_'s (different URIs):
.. _target: first
.. _target: second
-: expected
+: output
Duplicate external
target_
's (different URIs):
<system_message backrefs="id1" level="2" line="5" source="test data" type="WARNING">
Duplicate explicit target name: "target".
- <system_message backrefs="id3" ids="id2" level="3" line="1" source="test data" type="ERROR">
? ^
+ <system_message backrefs="id3" ids="id2" level="3" line="2" source="test data" type="ERROR">
? ^
Duplicate target name, cannot be used as a unique reference: "target".
and this
input:
Unknown reference_.
-: expected
+: output
Unknown
reference_
.
- <system_message backrefs="id2" ids="id1" level="3" line="1" source="test data" type="ERROR">
? ---------
+ <system_message backrefs="id2" ids="id1" level="3" source="test data" type="ERROR">
Unknown target name: "reference".
and the like...
Still 6 failures...
---
docutils/docutils/parsers/rst/states.py | 1 +
1 files changed, 1 insertions(+), 0 deletions(-)
commenter: kirr79
posted: 2010-03-01 17:09:04
title: #67 latex2e: recorddeps to images rel to CWD (+ lots of fixes)
Subject: [PATCH 6/8] rst: teach Inliner to propagate .line for footnote_reference elements
This fixes 3 failures like
input:
Duplicate manual footnote labels, with reference ([1]_):
.. [1] Footnote.
.. [1] Footnote.
-: expected
+: output
Duplicate manual footnote labels, with reference (
[1]_
):
1
Footnote.
1
<system_message backrefs="id3" level="2" line="5" source="test data" type="WARNING">
Duplicate explicit target name: "1".
Footnote.
- <system_message backrefs="id5" ids="id4" level="3" line="1" source="test data" type="ERROR">
? ^
+ <system_message backrefs="id5" ids="id4" level="3" line="2" source="test data" type="ERROR">
? ^
Duplicate target name, cannot be used as a unique reference: "1".
And also, as I promised, this restores line info in ERROR where file was
incorrect.
Still 3 failures...
commenter: kirr79
posted: 2010-03-01 17:09:20
title: #67 latex2e: recorddeps to images rel to CWD (+ lots of fixes)
Subject: [PATCH 7/8] rst: teach Inliner to propagate .line in interpreted_or_phrase_ref()
This fixes 3 last failures like
test_functional.py: functional/tests/standalone_rst_pseudoxml.py; test (test_functional.FunctionalTestCase):
--- functional/expected/standalone_rst_pseudoxml.txt
+++ functional/output/standalone_rst_pseudoxml.txt
@@ -1884,7 +1884,7 @@
<system_message backrefs="id82" ids="id81" level="3" line="372" source="functional/input/data/standard.txt" type="ERROR">
Unknown target name: "nonexistent".
- <system_message backrefs="id84" ids="id83" level="3" line="399" source="functional/input/data/standard.txt" type="ERROR">
+ <system_message backrefs="id84" ids="id83" level="3" line="398" source="functional/input/data/standard.txt" type="ERROR">
i.e. we've got all that line numbers destabilized by Nodes.setup_child()
change back.
---
docutils/docutils/parsers/rst/states.py | 7 +++++--
1 files changed, 5 insertions(+), 2 deletions(-)
commenter: kirr79
posted: 2010-03-01 17:09:39
title: #67 latex2e: recorddeps to images rel to CWD (+ lots of fixes)
Subject: [PATCH 8/8] latex2e: record_dependencies to images relative to CWD, not document source
So that e.g. for
--- path/to/1.txt ---
.. image:: pics/start.png
---------------------
rst2latex --record-dependencies path/to/1.txt
dependencies are
path/to/pics/start.png
not
pics/start.png
This is the same behaviour we have in Include directive and probably
others, e.g. for
--- path/to/3.txt ---
.. include:: 4.txt
---------------------
rst2 --record-dependencies path/to/3.txt
dependencies are
path/to/4.txt
This is the only consistent way which is meaningful for make(1), because
if otherwise if we treat paths and "urls" in record-dependencies
differently, the result is not suitable for project builds.
--------
Please pay attention, that this patch won't work without all ther other
patches I posted before. The problem was there was no good accounting
for source/line info in Docutils -- lot's of nodes ended up being
created and processed without .source / .line attributes, and this
creates problem at ``node.source``
---
docutils/docutils/writers/latex2e/__init__.py | 5 ++++-
docutils/test/test_dependencies.py | 3 +--
2 files changed, 5 insertions(+), 3 deletions(-)
commenter: kirr79
posted: 2010-03-01 17:10:07
title: #67 latex2e: recorddeps to images rel to CWD (+ lots of fixes)
attachments:
commenter: kirr79
posted: 2010-03-01 17:10:46
title: #67 latex2e: recorddeps to images rel to CWD (+ lots of fixes)
attachments:
commenter: kirr79
posted: 2010-03-01 17:11:27
title: #67 latex2e: recorddeps to images rel to CWD (+ lots of fixes)
attachments:
commenter: kirr79
posted: 2010-03-01 17:11:48
title: #67 latex2e: recorddeps to images rel to CWD (+ lots of fixes)
attachments:
commenter: kirr79
posted: 2010-03-03 18:11:17
title: #67 latex2e: recorddeps to images rel to CWD (+ lots of fixes)
attachments: 6B61
commenter: kirr79
posted: 2010-03-03 18:11:41
title: #67 latex2e: recorddeps to images rel to CWD (+ lots of fixes)
attachments:
commenter: kirr79
posted: 2010-03-03 18:12:27
title: #67 latex2e: recorddeps to images rel to CWD (+ lots of fixes)
I've updated patches 4 and 6 to apply cleanly to today's trunk (resolved conflicts after recent SVG & SWF support addition).
The text was updated successfully, but these errors were encountered: