Software! Math! Data! The blog of R. Sean Bowman
The blog of R. Sean Bowman
June 05 2015

Here’s another example of a project I used the Redo build system for. Unlike the previous application, which was for a C++ project, this one is for a short book I was writing on derivatives in calculus. (I don’t think I ever finished it, though…) This example highlights the flexibility of Redo as a build system.

I was experimenting with using LaTeX, a customized version of PlasTeX, the calibre software suite, and some of my own custom software to make books in .mobi format that contained nice looking math. (You can see the books I finished here. It’s an old picture! Don’t make fun!) As an aside, the plasTeX software back then was in a much worse state than it is now, which is why I had to modify it. I’d like to revisit it, as it looks like they’re really made some substantial changes.

Building the HTML book

Anyhow, building these books is a pain. There are figures that have to be made, the LaTeX code gets converted to HTML, which is munged and then converted to MOBI… it’s really quite an ordeal. Redo helped me automate everything pretty easily. Let’s see how. First, we need to make the “book” itself, using book.do.

for f in plots/*.r; do
    FIG=${f#plots/}
    echo figures/${FIG%.r}.png
done | xargs redo-ifchange
redo-ifchange derivatives.tex create_ebook.py
plastex --theme=minimal --output-encoding=ascii\
  --image-resolution=160 derivatives.tex 2>&1 >/dev/null
cd derivatives
python ../create_ebook.py derivatives_ebook.html
echo "don't forget to check labels!!!" >&2

The PNG figures are all stored in a figures directory, while the R scripts for making them are in the plots directory. The first thing we do is make sure all the figures are up to date. We loop through the scripts in the plots directory (each corresponds to one output PNG), taking the basename of the file and echoing out that plus the extension .png. This is piped (using xargs) to redo-ifchange. So, if any of the PNGs are out of date, or a new R script is added, an updated figure is automatically generated. Again, I want to emphasize

Automatic Dependencies

Redo lets you easily specify your dependencies however you like. In this case, we use the shell for loop. We could have also used find or written our .do script in a language like Python, using its features for listing files. If doesn’t matter – listing dependencies in Redo is easy, even if you don’t quite know what your dependencies are!

From there it’s pretty easy: we also depend on the .tex source as well as a python file which does some rearranging of the HTML output. We run plastex, run the python file, and then remind ourselves to check something. (I don’t quite remember what the issue was, but there were occasional problems that I needed to check by hand.)

Making figures

Now how do we actually make the PNG files from the scripts in the plots directory? Here’s were we can use a default rule, default.png.do.

FNAME=${2#figures/}
redo-ifchange plots/$FNAME.r plots/common.rlib
cd plots;
Rscript $FNAME.r ../$3 > /dev/null
mogrify -trim ../$3

We get the basename of the PNG, and note that it depends both on the R script used to make it as well as a library of common utilities I set up to make things easier. We cd in to the plots directory, run Rscript (which is a batch version of R), outputting to ../$3. Remember back from book.do that $3 here will have the form figures/filename.png, so we’re putting it in the right place. Finally, we use the ImageMagick tools to get rid of some extra space.

Converting to MOBI

To convert the HTML output created by book.do to mobi format, there’s mobi.do. This just uses the Calibre ebook-convert program, and depends on the book being up to date.

redo-ifchange book
ebook-convert limits/derivatives_ebook.html\
  derivatives.mobi --cover cover.jpg\
  --authors "R. Sean Bowman" --level1-toc "//h:h1"\
  --language en > /dev/null

Testing on my Kindle

Finally (and I think this is pretty cool, although maybe it doesn’t have so much to do with Redo…), there’s an action to upload the current version to my kindle for testing. It’s called upload.do, and it just uses the calibre-smtp program to send mail. It looks something like this, although I’ve obscured some details for anonymity.

redo-ifchange mobi
calibre-smtp --attachment limits.mobi\
  --relay smtp.gmail.com --port 587\
  --username (my username) --password (my passwd)\
  --encryption-method TLS (from email) (to email) ""

So after I’ve written a new chapter, I can just type redo upload and redo will make by book (including all the figures) and upload it to my kindle so that I can see what it looks like right then. How cool! For my first book I did lots of this by hand or wrote python scripts to handle it. But Redo only builds what’s necessary, and as we’ve seen, it’s quite easy to use. Cool!

Approx. 798 words, plus code