Is there a plan to make the macro `print!` work on notebook? #349

baiguoname · 2023-11-21T10:43:02Z

baiguoname
Nov 21, 2023

As mentioned on #303 and this, a progress bar is really important for someone doing data analysis on a notebook. If the reason why this feature is not implmented yet is lack of time, you can give me some guidances and I will give it a try.

davidlattimore · 2023-11-22T00:14:42Z

davidlattimore
Nov 22, 2023
Maintainer

Thanks for volunteering to look a this! I think the first thing we need to do is some design work. I'm not 100% sure how it should work, but we can discuss options and figure it out.

If the goal is progress bars in a notebook, then one alternative to making print! work would be to define and implement APIs that allow Rust code running in the notebook to communicate with Javascript code running in the browser. This would allow implementation not only of things like progress bars, but also interactive controls. e.g. some inputs that let you change some values and when you change them the output updates. This would be awesome, since you wouldn't need to wait for compilation. The controls could feed updates to the already compiled and running Rust code. This is something that I've been wanting to do for quite some time, but I have too many projects and not enough time. I think it's something that Jupyter notebooks using Python can do.

For making print!, or something equivalent to it work, there are few things to think about. The main reason why it doesn't work currently is because the output of the subprocess is captures a line at a time so that we can look for special control lines. For example if the code being run panics, then it prints out EVCXR_PANIC_NOTIFICATION to notify the parent process that a panic occurred. Putting these on stdout was probably a mistake. I should have just used a unix socket, but at the time, I wanted to make porting to non-unix platforms easier and didn't realise that recent versions of Windows actually do support unix sockets. The internal commands could be moved to a socket, however evcxr also allows user code to make use of some of these. e.g. printing EVCXR_BEGIN_CONTENT text/html\n....\nEVCXR_END_CONTENT\n to print HTML for display in the browser. We can't move that off stdout without breaking backward compatibility.

There are a number of crates on crates.io that integrate with evcxr for this kind of output. So I feel that it's important to maintain backward compatibility with this way of outputting HTML, even if it's perhaps inconvenient.

One option, would be to allow outputting of multiple content blocks. So for example, right now the following doesn't really work as you'd want it to:

for i in 0..10 {
    println!("EVCXR_BEGIN_CONTENT text/html\n<p>Foo {i}</p>\nEVCXR_END_CONTENT");
    std::thread::sleep(std::time::Duration::from_millis(500));
}

Evcxr currently collects these up and each text/html block overwrites the previous. When execution finishes, it prints only the last HTML block. It should probably instead print all the HTML blocks and do so as they're emitted, not wait for execution to finish.

At present, code can output both a text/html block and a text/plain block. The jupyter notebook will show only the HTML and the REPL will show only the text/plain. In order to preserve this, we'd probably need to have the notebook suppress output of text/plain if it has seen text/html output.

We could then extend the block format to allow replacement of previous output. One possibility would be to allow an optional modifier after the end marker. e.g. EVCXR_END_CONTENT REPLACE. Although some investigation of the Jupyter protocol would need to be done to see what's possible.

For actually just making print! work, I think we'd need to change the code that reads from the subprocess to not be line-based. When it successfully reads some content, any lines, or partial lines that don't start with "EVCXR" or some prefix of "EVCXR" if we've only got a few characters could be passed through. This would mean that print!("E") wouldn't be passed through until we received more content. That's kind of ugly, but probably unlikely to be a big problem in practice. I guess if we're worried about that, we could try to read more from the subprocess with a shortish timeout of a few milliseconds and we don't receive any more, assume that it's not an EVCXR control command and pass it through.

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

Is there a plan to make the macro `print!` work on notebook? #349

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{editor}}'s edit

{{editor}}'s edit

Uh oh!

Replies: 1 comment

Uh oh!

{{title}}

Uh oh!

Select a reply

Uh oh!

Uh oh!

Is there a plan to make the macro print! work on notebook? #349

Uh oh!

Uh oh!

baiguoname Nov 21, 2023

Replies: 1 comment

Uh oh!

davidlattimore Nov 22, 2023 Maintainer

Is there a plan to make the macro `print!` work on notebook? #349

baiguoname
Nov 21, 2023

davidlattimore
Nov 22, 2023
Maintainer