Announcement

Collapse
No announcement yet.
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • A new programme for literate programming/scientific research

    Hi
    Thanks to Christoffer Baum a new programme for literate programming/scientific research is present at the SSC.

    It is called log2markup and with simple markings during logging, log2markup takes such a generated text log file in text format and turn the log output into a document integrating text, commands and command outputs.

    If the writer has used markup like eg markdown, LaTex or HTML the document is ready for use as such.

    Besides log2markup there are a set of commands/programs that integrates text and output into one document (alfabetic order):The new command log2markup are in many ways very similar to the above.
    It differs from the above commands in following way:
    • texdoc and StatWeave integrates LaTex text with Stata commands and outputs. However the learning curve for LaTex is quite high. And worse the writeability and readability of LaTex is bad (the authors opinion)
    • markdoc and and weaver can be based on either of the markup languages markdown, HTML or LaTex or a combination of markdown and one of the other two.The difference to log2markup is that they have integrated the use of CSS, Pandocand HTML printer into the code
    One of the advantages of log2markup is that if you use tools like eg estout that can produce output in markup like LaTex or HTML then if such an output appears together with text written in the same markup then it all will be fully integrated without anything else than a special prefix before that command (see first example below).
    The prefix means that the command is not shown and the output is considered as a text block

    Below is an example. Here the output is directed to the log. Usually outputs are saved in a file with another extension that log.

    I'll recommend that some of these are tried out to see which ones fit your needs the best.

    Enjoy!!

    Example: Markdown and HTML mixed
    Code:
    . ********************************************************************************
    . *** HTML test file *************************************************************
    . ********************************************************************************
    . 
    . capture log close
    
    . set linesize 250 // The log should not break output lines
    
    . log using log2markup_test.log, replace
    -------------------------------------------------------------------------------------------------
          name:  <unnamed>
           log:  C:\Users\NHBR\Documents\stata\ado\log2markup\log2markup_test.log
      log type:  text
     opened on:  19 Feb 2016, 07:47:07
    
    . 
    . /***
    > <html>
    > <head>
    > </head>
    > <body>
    > ***/
    . 
    . /***
    > # Document demonstrating the use of log2markup
    > 
    > Note that html or latex can be inserted as plain text for later transformation.
    > ***/
    . 
    . //OFF
    . /***
    > Text blocks inside `//OFF` and `//ON` is ignored.
    > 
    > Below is the code for generating the data for the estout table:
    > ***/
    . sysuse auto, clear
    (1978 Automobile Data)
    
    . replace price = price / 1000
    variable price was int now float
    (74 real changes made)
    
    . replace weight = weight / 1000
    variable weight was int now float
    (74 real changes made)
    
    . 
    . quietly regress price weight mpg
    
    . estimates store m1, title(Model 1)
    
    . 
    . generate forXmpg = foreign * mpg
    
    . 
    . quietly regress price weight mpg forXmpg foreign
    
    . estimates store m2, title(Model 2)
    
    . 
    . label variable foreign "Foreign car type"
    
    . label variable forXmpg "Foreign*Mileage"
    
    . //ON
    . 
    . /***### A demonstration on integrating Markdown text blocks with Stata commands
    > 
    > The integration with Stata is demonstrated with an example from the command 
    > estout. 
    > 
    > The output from estout can be written in eg html or Latex. And that will be 
    > exploited further below.
    > 
    > The code leading to the data are hidden in the do file together with a comment 
    > between a `//OFF` and a `//ON`. This way the same document keeps all information 
    > but only show what is needed.
    > 
    > If a command is prefixed with `/**/` only the command is shown:
    > ***/
    . 
    . /**/estout      *, cells(b(fmt(%9.3f)) se(par))                         ///
    > stats(r2_a N, fmt(%9.3f %9.0g) labels(R-squared))       ///
    > legend label collabels(none) varlabels(_cons Constant) style(html)
    
    <tr><td>                    </td><td>     Model 1</td><td>     Model 2</td></tr>
    <tr><td>Weight (lbs.)       </td><td>       1.747</td><td>       4.614</td></tr>
    <tr><td>                    </td><td>     (0.641)</td><td>     (0.725)</td></tr>
    <tr><td>Mileage (mpg)       </td><td>      -0.050</td><td>       0.263</td></tr>
    <tr><td>                    </td><td>     (0.086)</td><td>     (0.111)</td></tr>
    <tr><td>Foreign*Mileage     </td><td>            </td><td>      -0.307</td></tr>
    <tr><td>                    </td><td>            </td><td>     (0.109)</td></tr>
    <tr><td>Foreign car type    </td><td>            </td><td>      11.240</td></tr>
    <tr><td>                    </td><td>            </td><td>     (2.752)</td></tr>
    <tr><td>Constant            </td><td>       1.946</td><td>     -14.450</td></tr>
    <tr><td>                    </td><td>     (3.597)</td><td>     (4.426)</td></tr>
    <tr><td>R-squared           </td><td>       0.273</td><td>       0.526</td></tr>
    <tr><td>N                   </td><td>          74</td><td>          74</td></tr>
    
    . 
    . /*** 
    > This the command demonstrating the 2 output examples just below.
    > 
    > The command prefix `/***/` will make only the output from the command is 
    > shown. 
    > 
    > But then the output will be marked as code, eg surrounded with 
    > `<pre><code>` and `</pre></code>` in html.
    > 
    > Output will look like:
    > ***/
    . 
    . /***/estout     *, cells(b(fmt(%9.3f)) se(par))                         ///
    > stats(r2_a N, fmt(%9.3f %9.0g) labels(R-squared))       ///
    > legend label collabels(none) varlabels(_cons Constant) style(html)
    
    <tr><td>                    </td><td>     Model 1</td><td>     Model 2</td></tr>
    <tr><td>Weight (lbs.)       </td><td>       1.747</td><td>       4.614</td></tr>
    <tr><td>                    </td><td>     (0.641)</td><td>     (0.725)</td></tr>
    <tr><td>Mileage (mpg)       </td><td>      -0.050</td><td>       0.263</td></tr>
    <tr><td>                    </td><td>     (0.086)</td><td>     (0.111)</td></tr>
    <tr><td>Foreign*Mileage     </td><td>            </td><td>      -0.307</td></tr>
    <tr><td>                    </td><td>            </td><td>     (0.109)</td></tr>
    <tr><td>Foreign car type    </td><td>            </td><td>      11.240</td></tr>
    <tr><td>                    </td><td>            </td><td>     (2.752)</td></tr>
    <tr><td>Constant            </td><td>       1.946</td><td>     -14.450</td></tr>
    <tr><td>                    </td><td>     (3.597)</td><td>     (4.426)</td></tr>
    <tr><td>R-squared           </td><td>       0.273</td><td>       0.526</td></tr>
    <tr><td>N                   </td><td>          74</td><td>          74</td></tr>
    
    . 
    . /***
    > When there are no prefix to a command both the command and the output above are
    > part of the markdown document.
    > 
    > The problem above is that the sorrounding html tags here are for code blocks.
    > 
    > But actually the code tags can be neglected by using the command prefix `/****/` 
    > instead.
    > 
    > Since html and latex can be inserted in markdown. It is at first considered as
    > plain text, but when the markdown are changed to html the inserted html becomes
    > active.
    > 
    > Now the table looks like:
    > ***/
    . 
    . /*** 
    > ***output only in pure form *** 
    > <table border="1">
    > <colgroup>
    > <col>
    > <col style="background-color:red">
    > <col style="background-color:yellow">
    > </colgroup>
    > ***/
    . 
    . /****/estout    *, cells(b(fmt(%9.3f)) se(par))                         ///
    > stats(r2_a N, fmt(%9.3f %9.0g) labels(R-squared))       ///
    > legend label collabels(none) varlabels(_cons Constant) style(html)
    
    <tr><td>                    </td><td>     Model 1</td><td>     Model 2</td></tr>
    <tr><td>Weight (lbs.)       </td><td>       1.747</td><td>       4.614</td></tr>
    <tr><td>                    </td><td>     (0.641)</td><td>     (0.725)</td></tr>
    <tr><td>Mileage (mpg)       </td><td>      -0.050</td><td>       0.263</td></tr>
    <tr><td>                    </td><td>     (0.086)</td><td>     (0.111)</td></tr>
    <tr><td>Foreign*Mileage     </td><td>            </td><td>      -0.307</td></tr>
    <tr><td>                    </td><td>            </td><td>     (0.109)</td></tr>
    <tr><td>Foreign car type    </td><td>            </td><td>      11.240</td></tr>
    <tr><td>                    </td><td>            </td><td>     (2.752)</td></tr>
    <tr><td>Constant            </td><td>       1.946</td><td>     -14.450</td></tr>
    <tr><td>                    </td><td>     (3.597)</td><td>     (4.426)</td></tr>
    <tr><td>R-squared           </td><td>       0.273</td><td>       0.526</td></tr>
    <tr><td>N                   </td><td>          74</td><td>          74</td></tr>
    
    . 
    . /***
    > </table>
    > ***/
    . 
    . /***
    > </body>
    > </html>
    > ***/
    . 
    . 
    . log close
          name:  <unnamed>
           log:  C:\Users\NHBR\Documents\stata\ado\log2markup\log2markup_test.log
      log type:  text
     closed on:  19 Feb 2016, 07:47:07
    ---------------------------------------------------------------------------------------------------
    
    . *** End of logging *************************************************************
    . 
    . /* Command log2markup converts the log into a markdown document */
    . log2markup using log2markup_test.log, log extension(md)
    And the output is:
    Code:
          +-------------------------------------------------------------------------------------+
        1 |                                                                             <html>  |
        2 |                                                                             <head>  |
        3 |                                                                            </head>  |
        4 |                                                                             <body>  |
        5 |                                                                                     |
        6 |                                                                                     |
        7 |                                     # Document demonstrating the use of log2markup  |
        8 |                                                                                     |
        9 |    Note that html or latex can be inserted as plain text for later transformation.  |
       10 |                                                                                     |
       11 |                                                                                     |
       12 |                                                                                     |
       13 |        ### A demonstration on integrating Markdown text blocks with Stata commands  |
       14 |                                                                                     |
       15 |       The integration with Stata is demonstrated with an example from the command   |
       16 |                                                                           estout.   |
       17 |                                                                                     |
       18 |       The output from estout can be written in eg html or Latex. And that will be   |
       19 |                                                           exploited further below.  |
       20 |                                                                                     |
       21 |    The code leading to the data are hidden in the do file together with a comment   |
       22 |  between a `//OFF` and a `//ON`. This way the same document keeps all information   |
       23 |                                                      but only show what is needed.  |
       24 |                                                                                     |
       25 |                    If a command is prefixed with `/**/` only the command is shown:  |
       26 |                                                                                     |
       27 |                                                                                     |
       28 |                                                                                ~~~  |
       29 |            estout      *, cells(b(fmt(%9.3f)) se(par))                         ///  |
       30 |                        stats(r2_a N, fmt(%9.3f %9.0g) labels(R-squared))       ///  |
       31 |                 legend label collabels(none) varlabels(_cons Constant) style(html)  |
       32 |                                                                               ~~~~  |
       33 |                                                                                     |
       34 |                                                                                     |
       35 |                   This the command demonstrating the 2 output examples just below.  |
       36 |                                                                                     |
       37 |          The command prefix `/***/` will make only the output from the command is   |
       38 |                                                                            shown.   |
       39 |                                                                                     |
       40 |                    But then the output will be marked as code, eg surrounded with   |
       41 |                                         `<pre><code>` and `</pre></code>` in html.  |
       42 |                                                                                     |
       43 |                                                             Output will look like:  |
       44 |                                                                                     |
       45 |                                                                                     |
       46 |                                                                                     |
       47 |                                                                              ~~~~~  |
       48 |                                                                                     |
       49 |   <tr><td>                    </td><td>     Model 1</td><td>     Model 2</td></tr>  |
       50 |   <tr><td>Weight (lbs.)       </td><td>       1.747</td><td>       4.614</td></tr>  |
       51 |   <tr><td>                    </td><td>     (0.641)</td><td>     (0.725)</td></tr>  |
       52 |   <tr><td>Mileage (mpg)       </td><td>      -0.050</td><td>       0.263</td></tr>  |
       53 |   <tr><td>                    </td><td>     (0.086)</td><td>     (0.111)</td></tr>  |
       54 |   <tr><td>Foreign*Mileage     </td><td>            </td><td>      -0.307</td></tr>  |
       55 |   <tr><td>                    </td><td>            </td><td>     (0.109)</td></tr>  |
       56 |   <tr><td>Foreign car type    </td><td>            </td><td>      11.240</td></tr>  |
       57 |   <tr><td>                    </td><td>            </td><td>     (2.752)</td></tr>  |
       58 |   <tr><td>Constant            </td><td>       1.946</td><td>     -14.450</td></tr>  |
       59 |   <tr><td>                    </td><td>     (3.597)</td><td>     (4.426)</td></tr>  |
       60 |   <tr><td>R-squared           </td><td>       0.273</td><td>       0.526</td></tr>  |
       61 |   <tr><td>N                   </td><td>          74</td><td>          74</td></tr>  |
       62 |                                                                                     |
       63 |                                                                             ~~~~~~  |
       64 |                                                                                     |
       65 |    When there are no prefix to a command both the command and the output above are  |
       66 |                                                     part of the markdown document.  |
       67 |                                                                                     |
       68 |      The problem above is that the sorrounding html tags here are for code blocks.  |
       69 |                                                                                     |
       70 |  But actually the code tags can be neglected by using the command prefix `/****/`   |
       71 |                                                                           instead.  |
       72 |                                                                                     |
       73 |     Since html and latex can be inserted in markdown. It is at first considered as  |
       74 |    plain text, but when the markdown are changed to html the inserted html becomes  |
       75 |                                                                            active.  |
       76 |                                                                                     |
       77 |                                                          Now the table looks like:  |
       78 |                                                                                     |
       79 |                                                                                     |
       80 |                                                                                     |
       81 |                                                                                     |
       82 |                                                   ***output only in pure form ***   |
       83 |                                                                 <table border="1">  |
       84 |                                                                         <colgroup>  |
       85 |                                                                              <col>  |
       86 |                                                 <col style="background-color:red">  |
       87 |                                              <col style="background-color:yellow">  |
       88 |                                                                        </colgroup>  |
       89 |                                                                                     |
       90 |                                                                                     |
       91 |                                                                                     |
       92 |                              <tr><td> </td><td> Model 1</td><td> Model 2</td></tr>  |
       93 |                     <tr><td>Weight (lbs.) </td><td> 1.747</td><td> 4.614</td></tr>  |
       94 |                              <tr><td> </td><td> (0.641)</td><td> (0.725)</td></tr>  |
       95 |                    <tr><td>Mileage (mpg) </td><td> -0.050</td><td> 0.263</td></tr>  |
       96 |                              <tr><td> </td><td> (0.086)</td><td> (0.111)</td></tr>  |
       97 |                       <tr><td>Foreign*Mileage </td><td> </td><td> -0.307</td></tr>  |
       98 |                                     <tr><td> </td><td> </td><td> (0.109)</td></tr>  |
       99 |                      <tr><td>Foreign car type </td><td> </td><td> 11.240</td></tr>  |
      100 |                                     <tr><td> </td><td> </td><td> (2.752)</td></tr>  |
      101 |                        <tr><td>Constant </td><td> 1.946</td><td> -14.450</td></tr>  |
      102 |                              <tr><td> </td><td> (3.597)</td><td> (4.426)</td></tr>  |
      103 |                         <tr><td>R-squared </td><td> 0.273</td><td> 0.526</td></tr>  |
      104 |                                       <tr><td>N </td><td> 74</td><td> 74</td></tr>  |
      105 |                                                                                     |
      106 |                                                                                  .  |
      107 |                                                                           </table>  |
      108 |                                                                                     |
      109 |                                                                                     |
      110 |                                                                            </body>  |
      111 |                                                                            </html>  |
      112 |                                                                                     |
      113 |                                                                                     |
          +-------------------------------------------------------------------------------------+
    Kind regards

    nhb

  • #2
    Why is the output right-aligned?

    Comment


    • #3
      It is because it is mata string colvector that is printed. That could be shown in better ways. I'll look into it.
      However the log option is mainly for testing the output in the log.
      If you do not set log option you'll get the similar allthough left aligned text in the output markdown file (here log2markup_test.md).

      If you then eg run pandoc on the md file to convert the md file to a full HTML ditto, eg
      Code:
      shell pandoc -o log2markup_test.html -f markdown -t HTML log2markup_test.md
      you'll be able see log2markup_test.html in your favorite browser.

      One of the differences to markdoc is that I've taken out all standardisation of pandoc as well as CSS and javascript settings (the last is only if you want an HTML output).
      This you have to set yourself when generating the document from log2markup.
      And it is of course a bit elaborous, but the benefit is that it (to me anyway) is easier eg to add LaTex like math into my document.
      Or to build slides through pandoc etc.
      Potential users are strongly urged to study the possibilities in pandoc.

      I'm working more examples to show the extend of the functionality of log2markup.
      Kind regards

      nhb

      Comment


      • #4
        Thanks to Kit Baum is there a new update of log2markup. There are only minor changes.

        log2markup is now integrated with the packages matrixtools and basetable.

        What's also new is that in time more and more examples will appear on my homepage for Stata.
        The homepage is mainly generated by markdown commented do files transformed by log2markup and then MkDocs

        So the homepage should be a living proof of the usability of log2markup, matrixtools and basetable.

        Enjoy
        Kind regards

        nhb

        Comment


        • #5
          Thanks to Kit Baum is there a new update of log2markup. There are only minor changes.
          Features are described at my homepage at log2markup.

          Further one can say that my homepage is a live demo of log2markup, since it is based on do files with comments in markdown.
          The log files from these do files are converted into markdown files and these are again transformed by mkdocs into the homepage content.
          Last edited by Niels Henrik Bruun; 17 Oct 2017, 00:35.
          Kind regards

          nhb

          Comment


          • #6
            Thanks to Kit Baum is there a new update of log2markup
            Kind regards

            nhb

            Comment


            • #7
              Thanks to Kit Baum is there a new update of log2markup.
              A wrapper -do2doc- for pandoc is added.
              Kind regards

              nhb

              Comment

              Working...
              X