Weave and markdoc packages

[Attaullah Shah]

I find weave and markdoc packages interesting for making notebooks. However, the markdoc package does not have an obvious option to control font style and size, while the weave package is great with one less appealing feature of writing div with every Stata command to pass the arguments to the dynamic document. This is specially the case when I develop DO files and send them over to someone else.
My questions are:
can we have a way of controlling font style and size in the markdoc package?
can we simply skip the div prefix with Stata command and still be able to pass commands and results to the dynamic document in weave package.

[haghish]

Weaver is a "log system", similar to smcl log, but instead created with HTML or LaTeX (version 3.3.0, published today). Weaver log is not autonomous as smcl and only registers what you wish to include. That's the idea. MarkDoc is based on smcl and thus it registers everything by default. But there are convenient ways to exclude commands or output from the dynamic document.

back to your question, MarkDoc allows you to edit the style, for sure. But it depends in what format you're exporting your document. If you are exporting a HTML or PDF format, you can use the CSS(filename) option to attach a CSS style sheet, where you have defined your font size, font family, etc. THis requires some level of familiarity with web languages, particularly HTML and CSS.

A relatively simpler alternative to write your document in LaTeX and export it to tex or pdf. The command would be like:

markdoc filename, export(tex) markup(latex)

[wbuchanan]

It should also be noted that this isn't a dynamic document solution. As haghish stated, it is a program that is built on the logging capabilities native to Stata. In other words, what you get in the document cannot be further manipulated without rebuilding the underlying log, and the program does not provide capabilities to communicate with Stata to allow the dynamic manipulation of cells. What you are continuing to refer to as a notebook is not a notebook, but rather a weaved document. There are differences between the two and major differences in the underlying computer architecture used to support them. If you need support for building a notebook interface, your best bet is to contribute to the existing work that has been done around Stata and Python integration, or to develop an independent Stata kernel for project Jupyter.

[haghish]

wbuchanan

It's actually incorrect. markdoc allows modification of the log file, although it's a post-processing system. You can "hide chunks of the log file", "Stata commands", or "output", without removing them from the SMCL log file. markdoc is only a single command and its biggest advantage is its simplicity. I see no point in developing software for dynamic documents that cannot be learned quickly. StatWeave is already a decent software for statistics geeks, but that's not what you wish the students work with.

I wrote Weaver because i don't think literate programming is the best solution. Weaver is a "HTML or LaTeX log file" that runs in parallel to Stata log-file. and as you work the analysis you can display what you're interested in the dynamic document interactively and view the document instantly because it's updated in real time.

Attaullah Shah

If you want to send a do-file to someone without forcing them to install the required software for Weaver, use the noprint option. It creates a HTML or LaTeX document without requiring any third-party software.

[Attaullah Shah]

haghish Thanks for your reply. I am more interested in using MS word, so can you suggest how to control the font size and style in markdoc while creating MS Word document.

[haghish]

wbuchanan

P.S. markdoc is a literate programming package and naturally, not a text editor. I'm not sure what are you referring to as notebook. something like iPython? if so, then I don't think that is a good idea because a natural workflow of Stata should be preserved, i.e. working in Stata do-file editor. For Weaver, I was very much tempted to create a "special text editor" that communicates with Stata and create the Weaver log without the prefix commands (div, img, txt, tble). But I disliked the idea of "another text editor" and don't think forcing Stata users to write their document in another text editor/software would be a good idea. Particularly, because it forces the users to create additional script files which increases the chance of error and problems with updating the script files.

I'm not sure what you mean by "the program does not provide capabilities to communicate with Stata to allow the dynamic manipulation of cells." Please explain what do you mean by this,.

You also mentioned that "what you get in the document cannot be further manipulated without rebuilding the underlying log." This is only true about markdoc. Because weaver allows preserving and restoring status of the log. Nevertheless, modifying the code in order to rebuild the dynamic document is the base of literate programming, where the document is written by code. I'm not sure if I understand what you are trying to note.

[haghish]

Attaullah Shah

markdoc uses Pandoc to convert Markdown to Docx. if you'd like to change the style of your docx dynamically, you should define a new template docx yourself and pass it to Pandoc. I haven't provided an option for that but I have tried it briefly a while ago and it works.

If you're interested to look into that, I can add the option to markdoc on an "experimental ado file" and email it to you. You could try it out and see if it's working properly. Does that interest you?

You'd also find this link useful:
http://stackoverflow.com/questions/1...mplex-template

[Attaullah Shah]

haghish I am very much interested and looking forward to it. You can email me at [email protected] or post it here for others to benefit from. Thanks for your help

[haghish]

Well let's see what you can do with it and if it works fine, I'll keep it in the next release. And thanks for your cooperation. ;-)

[Attaullah Shah]

haghish I think I have worked out a solution, now waiting for your modified version of markdoc

[haghish]

Attaullah Shah

Perfect! Could you also share some examples and say what could be changed with the template docx file?

The time I tried the docx templates I was trying to reduce the left and right margins of the document exported by Pandoc. But It seems that I could not get Pandoc to recognize the margins of the template. I wonder if you tried anything with the margins?

[Attaullah Shah]

In the current folder where the cmd screen shows the location, I have placed a document with the name reference.docx where I have made changes to the paragraph style named as "First Paragraph" as follows
font = Courier new
Font size = 9
background colour = light yellow
manual margin setting
the pandoc command to execute is

pandoc --reference-docx=reference.docx -o newdoc.docx

Whatever I then type in the cmd window becomes part of the newdoc.docx and is automatically formatted as the setting in reference.docx. The reference.docx is attached for your review.

[haghish]

What is very important is the left and right margins. Could you please make a reproducible example? You may follow this procedure:

1. use MarkDoc to create a Markdown file. To do so, leave the export() option empty
2. Use the markdown file with the template you made in Pandoc to convert the Markdown to docx
3. Does Pandoc apply the margins in the new document, generated from the source Markdown?

If yes, please also specify your operating system and Pandoc version (did you use the automatic installation?).

Thanks for your effort.

[haghish]

Attaullah Shah

nevermind, I figured it out and will update the "template" option in the next release. This would take some time though, because a template option should support all other document formats (HTML, LaTeX, ePub, Docx, ODT). The one that demands some programming is the LaTeX because it's not based on Pandoc...

Thanks for the initiative. This was a step forward.

[wbuchanan]

haghish I think the program you've developed is good, but like you said it is a post-processing program. Notebooks are dynamic documents that preserve the code and output and allow the output to be modified dynamically (e.g., you move to a cell and can update the computations/processes that occur in that cell in isolation from the rest of the notebook or can run the entire document again). The markdown package in R does not provide notebook capabilities either. Notebooks are by their nature meant to be interactive (e.g., used like the command prompt), but markdoc and the R analog are not and were intentionally designed not to do that.