MarkDoc (GitHub) bug: Cannot use directory paths on tex for Windows

[Anders Alexandersson]

I am unable to use the user-written program markdoc (GitHub) for directory paths on tex. I am using MarkDoc version 3.9.2 on Windows 7.

I first reported markdoc to be incompatible with cd "C:/temp" on version 3.8.7 here: http://www.statalist.org/forums/forum/general-stata-discussion/general/1355554-markdoc-manual-gui?p=1364009#post1364009

A claimed "bug fix" for version 3.9.1 was reported here: http://www.statalist.org/forums/foru...83#post1366083

The example I used for 3.9.1 has a different problem of outputting wrong tex in version 3.9.2. This is the example code I reported:

log using example, replace

sysuse auto, clear

/***
Part A.
***/

describe

/***
Part B.
***/

di "hello world"


log c

My calling code (with the simple style) is:

markdoc "F:\DOH\tmp2_1dec.do" , markup(markdown) export(tex) replace statax ///
     style("simple") master toc noisily

I used a new do file name to allow further testing. The describe command outputs wrong tex path:

Contains data from C:\\Program Files (x86)\Stata14\ado\base/a/auto.dta

The above text code does not compile. The error is

! Undefined control sequence.
l.39 ...a from C:\\Program Files (x86)\Stata14\ado

\base/a/auto.dta

I can manually edit the tex code to make it run and produce the wanted PDF. This tex code works:

Contains data from C:/Program Files (x86)/Stata14/ado/base/a/auto.dta

The problem used to affect only stata style but now also affects the simple style. With simple style the last part of the Stata output is:

Executing Pandoc Command

"c:\ado\plus\Weaver\pandoc\pandoc.exe" --mathjax --toc  "C:\Users\AALEXA~1\AppData\Local\Temp\ST_0000000b.
> tmp.md" -o "C:\Users\AALEXA~1\AppData\Local\Temp\ST_00000009.tmp.tex"
::Part A. Comment about describe.

1::      . di "hello world"
::
(note: file C:\Users\AALEXA~1\AppData\Local\Temp\ST_0000000m.tmp not found)
(MarkDoc created F:\DOH\tmp2_1dec.tex)

which looks weird, and the tex output is:

! LaTeX Error: Something's wrong--perhaps a missing \item.




See the LaTeX manual or LaTeX Companion for explanation.

Type H <return> for immediate help.

...



l.2 ...n}{\numberline {1}References}{1}{section.1}

The usual reply has been something like "this has been fixed in the new version". There is already a new version, 3.9.3. But this was supposed to be fixed in version 3.9.2, as reported here:
http://www.statalist.org/forums/foru...36#post1366236

I am willing to send my full test file off-line if needed.

[Anders Alexandersson]

Version 3.9.3 makes the inconsistent bug in version 3.9.2 consistent for style("simple") and style("stata"). Since this MarkDoc issue affect tex, I should mention that my *tex software is TexWorks 0.4.5 r.1280 (MikTex 2.9 64-bit).

The bug now, regardless of style, is as I reported in #1 for style("stata"). That is, describe outputs the wrong tex code for the directory:

Contains data from C:\\Program Files (x86)\Stata14\ado\base/a/auto.dta

This is a smaller problem than before, so partially good news. For completeness, here is a slightly more realistic example. Noteworthy is the extra first line for changing directory, cd "F:/doh":

cd "F:/doh"
/**/ qui log using example, replace

sysuse auto, clear

/***
Part A. Comment about `describe`.
***/

describe

/***
Part B. Comment about `display`.
***/

di "hello world"

qui log c

The tex code for the first line is:

F:\\doh

For version 3.9.1, the tex output was F:/doh/. Both code snippets are valid tex code. I mention it out because it can matter if you wrongly assume what type of slash tex outputs. Therefore, please test this more realistic example. I used the F: drive but any other drive is fine for testing, of course.

[Anders Alexandersson]

MarkDoc version 3.9.4 has been announced but the bug remains as in version 3.9.3.

haghish Have you added code for testing the bug? If so, where is the code? I suggest you add cd and describe (like I did) as part of "rundoc_example.do". That way, the bug will be easier to test since new no new code for testing the bug will be required in future releases.

[Anders Alexandersson]

I recommend using older version 3.8.7 if you experience the bug.

As an aside, I found one more small tex bug in version 3.8.7 for sts list. It creates LaTex code

\begin{longtable}[]{@{}c@{}}

But Texworks shows this as an error.

! LaTeX Error: Environment longtable undefined.

See the LaTeX manual or LaTeX Companion for explanation.
Type H <return> for immediate help.
...

l.1481 \begin{longtable}
[]{@{}c@{}}
?

Also, I found several more bugs in current version 3.9.4 for math notation. For example, output of $ $ instead of $$, that is wrong extra empty LaTeX space. I rather not spend more time on troubleshooting the current release until it works better for me than 3.8.7 does.

[Anders Alexandersson]

I found a simple fix for the LaTeX error "Environment longtable undefined" that I experienced in MarkDoc version 3.8.7. The fix is to add this line of LaTeX code in the "preamble", that is, at at the top of the .tex document where the other environments are defined in a similar way:

\usepackage{longtable}

[Anders Alexandersson]

The root of the LaTeX error "Environment longtable undefined" turned out to be paging. The table spanned over two pages for which longtable is a quick fix. However, the Stata Journal LaTeX style fix instead is to use either \clearpage or \cnp. Paging is a root problem of LaTeX itself. There were some other minor LaTeX differences for the table which is symptomatic.

My original posting has a bad link. I first reported markdoc to be incompatible with cd "C:/temp" on version 3.8.7 here (in post #16): http://www.statalist.org/forums/foru...09#post1364009

haghish Ben Jann's alternative command texdoc (SJ 16-2) calls sjlog and I have not heard of similar problems in texdoc. I still see a huge putential with MarkDoc because Markdown is much simpler than LaTeX. The only robust bug fix seems to be to either call sjlog (e.g., similar to how texdoc does it) or to drop the the style(stata).

[haghish]

Unfortunately I cannot get "any" of the errors you are mentioning, not only on Mac, but also Windows 10 and 7. On the one hand, having bug reports are very valuable, making the software work better. On the other hand, if the reports are not well-organized, they won't be very helpful, but confusing. So I suggest you to discuss "one of the errors at the time" and move on to the next one, when the error is solved.

And please avoid writing plenty of posts one after another. It is really confusing. It'd be much better to gather all your information and then write a complete bug report. I have been able to assist those who touch on a specific problem and provide the do file much faster and more efficient. So, one problem at the time.

Reinstall MarkDoc and then report the biggest problem you have found that you think it should be solved first.

[Anders Alexandersson]

I was able to reproduce on a second PC the LaTeX output problem from example.do that I reported in this thread in post #2. I am using Stata 14.2 and markdoc version 3.8.7 and MikTex/TeXworks also on PC #2.

PC #2 uses the directory "C:\temp\md". Therefore, this is the code I used on PC2 to call example.do:

markdoc "C:\temp\md\example.do" , markup(markdown) export(tex) replace statax ///
     style("stata") master toc noisily

This the LaTeX output from running the do file:

\documentclass{article}
\usepackage[article,notstatapress]{sj}
\usepackage{epsfig}
\usepackage{stata}
\usepackage{booktabs}         %for tables
\usepackage{hyperref}         %use hyperlink
\usepackage{shadow}
\usepackage{natbib}
\usepackage{chapterbib}
\providecommand{\tightlist}{\setlength{\itemsep}{0pt}\setlength{\parskip}{0pt}}\bibpunct{(}{)}{;}{a}{}{,}

\begin{document}
\clearpage
\tableofcontents
\clearpage

\inserttype[st0001]{article}
\author{Short article author list}{    \and}
\title[Short toc article title]{}
\maketitle

\begin{stlog}
      . cd "c:/temp"
      c:\temp
      
      
      .  sysuse auto, clear
      (1978 Automobile Data)
      
      
\end{stlog}

Part A. Comment about \texttt{describe}.

\begin{stlog}
      .  describe
      
      Contains data from C:\Program Files (x86)\Stata14\ado\base/a/auto.dta
        obs:            74                          1978 Automobile Data
       vars:            12                          13 Apr 2014 17:45
       size:         3,182                          (_dta has notes)
      ----------------------------------------------------------------------------------------------
                    storage   display    value
      variable name   type    format     label      variable label
      ----------------------------------------------------------------------------------------------
      make            str18   %-18s                 Make and Model
      price           int     %8.0gc                Price
      mpg             int     %8.0g                 Mileage (mpg)
      rep78           int     %8.0g                 Repair Record 1978
      headroom        float   %6.1f                 Headroom (in.)
      trunk           int     %8.0g                 Trunk space (cu. ft.)
      weight          int     %8.0gc                Weight (lbs.)
      length          int     %8.0g                 Length (in.)
      turn            int     %8.0g                 Turn Circle (ft.)
      displacement    int     %8.0g                 Displacement (cu. in.)
      gear_ratio      float   %6.2f                 Gear Ratio
      foreign         byte    %8.0g      origin     Car type
      ----------------------------------------------------------------------------------------------
      Sorted by: foreign
      
      
\end{stlog}

Part B. Comment about \texttt{display}.

\begin{stlog}
      .  di "hello world"
      hello world
      
      
      
\end{stlog}
\bibliographystyle{sj}
\bibliography{sj}
%\begin{aboutauthors}
%% Write some background information about the author(s).
%\end{aboutauthors}

\end{document}

The LaTex code has two errors, as seen when you compile the LaTeX code:

! Undefined control sequence.
l.25       c:\temp

! Undefined control sequence.
l.39       Contains data from C:\Program
                                         Files (x86)\Stata14\ado\base/a/auto...

I fixed the LaTeX errors above as before. The table output, however, is simple plain-text style looking despite that I specified style("stata"). The biggest problem that I think should be solved first is the lack of a reproducible example in the documentation that addresses the features and limitations of the stata style. For example, do you get simple plain-text table output (like the example on page 22 in the PDF journal article) or nice, clean table output like in Stata Journal? Another example, what output do you get for directories in Windows? I have showed two bug reports with code and output of a version-specific (3.8.7) and OS-specfic (Windows 7) problem of the stata style for directories whereas the documentation has no example, and you do not show code here on Statalist either. Do you have a backward slash or forward slash or some combination to handle directory output of the stata style on Windows? There is no knowing without you telling. Recent versions are too buggy for me to even be worthy further testing -- unless you address the problem with code and output like I did. Of the two examples I just gave -- table output quality and the LaTex errors -- the bigger problem to me is the simple table output quality even with the stata style being specified. Thank you for trying to help.

[Anders Alexandersson]

like the example on page 22 in the PDF journal article)

Correction of typo: The example is on page 18 (not on page 22). If my bug report is insufficient, please state what is missing or how it should look like.

[haghish]

Reinstall MarkDoc and then report the biggest problem you have found that you think it should be solved first.

MarkDoc is currently having pre-release of version 3.9.5... I recommend you to follow the package on GitHub to avoid old bugs. This happens most of the times, when bugs are due to old versions.

[Anders Alexandersson]

haghish Thank you. I am one of seven users who follow the package on GitHub. I reinstalled version 3.8.7 because later versions have introduced more bugs for the option style("stata") on Windows, in my experience, not less. Unless you or someone else show the LaTex output of an example.do file like I provided in post #2 we cannot make progress on my bug report, I suspect. As I wrote:

The biggest problem that I think should be solved first is the lack of a reproducible example in the documentation that addresses the features and limitations of the stata style.

Preferably in the next release 3.9.5

[Anders Alexandersson]

The new markdoc version 3.9.5 fixed the bug by adding an extra backward slash in the tex file. An alternative solution would have been to use forward slashes. I am still using Stata 14.2 on Windows 7 and TeXworks version 0.4.5 r.1280 (MikTeX 2.9 64-bit).

For future reference, this is the LaTeX output of the example on my work PC when I use markdoc version 3.9.5.

\documentclass{article}
\include{Statax}              %included in your working dorectory
\usepackage{epsfig}
\usepackage{hyperref}         %use hyperlink
\usepackage{booktabs}         %for tables
\usepackage{longtable}        %for long tables
\usepackage{fixltx2e}         % for \textsubscript
\usepackage{graphicx}
\usepackage[article,notstatapress]{sj}
\usepackage{stata}
\usepackage{shadow}
\usepackage{natbib}
\usepackage{chapterbib}
\providecommand{\tightlist}{\setlength{\itemsep}{0pt}\setlength{\parskip}{0pt}}\bibpunct{(}{)}{;}{a}{}{,}

\begin{document}
\clearpage
\tableofcontents
\clearpage

\inserttype[st0001]{article}
\author{Short article author list}{    \and}
\title[Short toc article title]{}
\maketitle

\begin{stlog}

      . cd "F:/doh"
      F:\\doh


      . sysuse auto, clear
      (1978 Automobile Data)
\end{stlog}

Part A. Comment about \texttt{describe}.

\begin{stlog}

      . describe

      Contains data from C:\\Program Files (x86)\\Stata14\\ado\\base/a/auto.dta
        obs:            74                          1978 Automobile Data
       vars:            12                          13 Apr 2014 17:45
       size:         3,182                          (_dta has notes)
      ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
                    storage   display    value
      variable name   type    format     label      variable label
      ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
      make            str18   %-18s                 Make and Model
      price           int     %8.0gc                Price
      mpg             int     %8.0g                 Mileage (mpg)
      rep78           int     %8.0g                 Repair Record 1978
      headroom        float   %6.1f                 Headroom (in.)
      trunk           int     %8.0g                 Trunk space (cu. ft.)
      weight          int     %8.0gc                Weight (lbs.)
      length          int     %8.0g                 Length (in.)
      turn            int     %8.0g                 Turn Circle (ft.)
      displacement    int     %8.0g                 Displacement (cu. in.)
      gear_ratio      float   %6.2f                 Gear Ratio
      foreign         byte    %8.0g      origin     Car type
      ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
      Sorted by: foreign
\end{stlog}

Part B. Comment about \texttt{display}.

\begin{stlog}

      . di "hello world"
      hello world

      . cd ..
      F:\\

      . cd "F:/doh"
      F:\\doh
\end{stlog}
\bibliographystyle{sj}
\bibliography{sj}
%\begin{aboutauthors}
%% Write some background information about the author(s).
%\end{aboutauthors}

\end{document}

Two related bugs remain: the old problem of only ASCII table output, and a new error "invalid syntax" in the dependency program markdocstyle.ado. I will soon (today or tomorrow) open two new threads for these two other problems.