Thursday, July 20, 2017

Installing ESA open type font in MikTeX


A post after 4 years of no rain, a new post ... this time about LaTeX and fonts.

Introduction


I came across a set of fonts to download that were available as *.otf files. Namely the cooperate identity fonts from the European Space Agency. These were available to download here: http://www.esa.int/identity/04T_fonts_01.html.

I wanted to use this font in an animated graphic I previously did with LaTeX -> dvips -> ps2pdf.
There mainly used the packages pst-solides3d and pstricks-add and animate. The animate package allows the creation of interactive movie like objects to be embedded in a pdf.

There are plenty of people that asked how to install (or use) such otf files with LaTeX. Almost all answers suggested the use of XeTeX or LuaTeX instead of the classical LaTeX or PdfTeX programs.
For example:

I did not further tried to make the animate package running with XeTeX because of the following comment on a stackexchange question:

https://tex.stackexchange.com/questions/340134/animate-with-pstricks-and-xelatex
Herbert (Which I think is Herbert Voß, a developer of PSTricks) left the following comment: 
use latex->dvips->ps2pdf. Animations are not really possible with xelatex
The  TeX system is based on using quite specific font description files and does not support otf files. Fonts that are available only as otf files have to be converted in something understandable by the TeX system. As far as I could see, XeTeX is first converting the otf files that have been installed to the system into a format that is understandable by the TeX system.
With some googling I came across the installfont package that should be able to convert otf fonts permanently for a TeX installation. This package is however not included with the MikTeX distribution but only with the TeX Live distribution. The intallfont package is a bash script that should convert and install otf files for a TeX system by calling appropriate programs. Of cause it failed, also after a lot of reconfiguration, renaming and what not. So every single step of what this script tries to do, had to be examined carefully. The steps taken follow:

Converting into Adobe font metric files

The first thing the fontinstall script does is to convert the otf files into tfm files. At that point I could not get any results that would render the font in question. So I was searching for some other conversion tool that may help. I found Fontlab's TransType which provided a 2 weeks trial installation. The tool looked great: I just dragged the otf files into the GUI and the font was shown in all 4 family members that were provided with the font. 

  • NotesEsaBol.otf
  • NotesEsaBolIta.otf
  • NotesEsaReg.otf
  • NotesEsaRegIta.otf


 The resulting files were adobe font metrics (afm):

  • NotesEsa.afm
  • NotesEsa-Bold.afm
  • NotesEsa-BoldItalic.afm
  • NotesEsa-Italic.afm 
To each afm file also a printer font binaries (pfb), information (inf), and printer font metric (pfm) file was created by the tool. Also: the tool provided more options to convert to; this included the true type font (ttf) format, that is also supported by the fontinstall script. I didn't get any usable results with either conversion in the beginning. In the end I used the pfb way, but I am not sure if its the best or recommend way.
    In the installfont script the tool otftotfm is used to create a tfm file from each otf file. Probably also a pfb file is generated with that too. After that it may create a afm file from every pfb file with the pf2afm tool.

    Rename the file pairs into Karl Berry scheme

    The installfont script tries to rename the files into the Karl Berry scheme based on keywords in the file name. E.g. if  "Bold" or "bold" is somewhere in the filename it would be renamed with the fontcut identifier bc. It is quite a big chunk of the script to do that. However the result was not quite what I would expect. Probably some keywords are not matching, e.g. Italic is not in the search list. So I renamed them by hand.
    What that Karl Berry scheme is, is explained in the documentation that comes with the fontinstall package. In short its like this:
           fxxy(y)zz.pfb
      f  - supplier (type foundry)
      xx - typeface  y(y) - weights/variants  (font cut)
      zz - font encoding
    Together f and xx form the font family. I chose the font family to be esa, thus "e" as the supplier and "sa" as the typeface. The font cut I chose according to the file e.g.: "bi" (bold italic) for the NotesEsa-BoldItalic.afm file.

    Here a link of the documention available at the time of writing: ftp://ftp.dante.de/tex-archive/support/installfont/installfont.pdf

    For example i found the Helvetica font in the MiKTeX installation under fonts\afm\adobe\helvetic with the filename phvb8a.afm. Thus "p" for Adobe, "hv" for Helvetica, "b" for bold and "8a" for AdobeStandardEncoding.

    In the fontinst package (which is not the same as the installfont package) documentation in doc\fontinst\manual\intro98.pdf, some other font encodings are listed. E.g. "8t" is symbolic for TeX 8-bit text (T1) encoding, and "8r" is symbolic for TeX 8-bit 'raw' (TeXBase1) encoding.
    An internet link to find it faster (at least at time of writing): http://www.let.rug.nl/alfa/tex/tetex30/fontinst/manual/intro98.pdf
    • NotesEsa.afm -> esar8a.afm
    • NotesEsa-Bold.afm -> esab8a.afm
    • NotesEsa-BoldItalic.afm -> esabi8a.afm
    • NotesEsa-Italic.afm -> esari8a.afm
    This renaming is necessary for the following step that uses the fontinst tex package to extract metrics from the afm files with the tex system. 

    Creating metrics and virtual fonts



      Next,  property list files, virtual property list files and font descriptions are extracted from the Adobe font metric (afm) files. This is done by the tex system itself. As such I used a tex file (an example from the fontinst manual) and compiled it with tex.

      The contents of the esa-drv.tex file:
      \input fontinst.sty
      \recordtransforms{esa-rec.tex}
      \latinfamily{esa}{}
      \endrecordtransforms
      \bye

      This file resembles a minimal file that combines a bunch of common stuff done. It searches for the font family in files that are located in the same directory beginning with "esa". From the remaining file name of the found files the fontcut and encoding is taken (assumed).

      After the command I ran the following command:
      >tex esa-drv.tex

      This created a bunch of new files:

      esab8a.afm ot1esa.fd esab8a.mtx esab8a.pl esab7t.vpl esa-drv.log
      esabi8a.afm t1esa.fd esab8r.mtx esab8r.pl esab8c.vpl esa-drv.dvi
      esar8a.afm ts1esa.fd esabi8a.mtx esabi8a.pl esab8t.vpl esa-rec.tex
      esari8a.afm 8resa.fd esabi8r.mtx esabi8r.pl esabc7t.vpl


      esabo8r.mtx esabo8r.pl esabc8t.vpl


      esar8a.mtx esar8a.pl esabi7t.vpl


      esar8r.mtx esar8r.pl esabi8c.vpl


      esari8a.mtx esari8a.pl esabi8t.vpl


      esari8r.mtx esari8r.pl esabo7t.vpl


      esaro8r.mtx esaro8r.pl esabo8c.vpl




      esabo8t.vpl




      esar7t.vpl




      esar8c.vpl




      esar8t.vpl




      esarc7t.vpl




      esarc8t.vpl




      esari7t.vpl




      esari8c.vpl




      esari8t.vpl




      esaro7t.vpl




      esaro8c.vpl




      esaro8t.vpl


      Note, the tex command did not run smoothly; a lot of errors warnings and the enter button needed to be pressed to continue.

      This included the generation of the font property list (pl) files of the orignal fontcuts in 8a and 8r encoding + two additional fontcuts: BoldOblique (bo) and RegularOblique (ro).  To each of those also a metrics (mtx) file was created. This is supposedly only a helper file of the fontinst package to make the afm files readable by TeX.Then there are a bunch of virtual font property list (vpl) files, that are probably generated fontcuts and encodings of the fontcuts that were provided. I would have expexted the bo and ro fontcuts to be also a vpl file though.

      A good source of information here is probably the book "Font & Encodings" from Yannis Haralambous. I only looked at some previews of it.
      https://books.google.nl/books?id=qrElYgVLDwYC&pg=PA281&lpg=PA281&dq=mtx+latex+file&source=bl&ots=4iwJt-20OO&sig=VVJxHCPNs8nFNkXza90DfmtUozE&hl=en&sa=X&ved=0ahUKEwin5cO_8obVAhXNUlAKHVGwDsgQ6AEIWTAJ#v=onepage&q=mtx%20latex%20file&f=false

      Creating TeX font metrics and virtual font files

      Here it got really confusing for me: so many files, nothing working, not testable. So I asked the almighty internet for help:
      https://tex.stackexchange.com/questions/375335/why-should-one-use-pltotf-and-vptovf-to-intall-a-font
      Next step generate TeX font metrics (tmf) files from the property list (pl) files with the pltotf tool. And then generate additional tmf files from the virtual font property list (vpl) files. Also virtual font (vf) files were generated from the

      for file in *.pl; do pltotf $file ${file%pl}tfm; done
      for file in *.vpl; do vptovf $file ${file%vpl}vf ${file%vpl}tfm; done
      
      
      Note that the commands are different from the tutorials, since the syntax of both tools require an explicit output file name.
      I did that from a cygwin terminal because the above is bash language that is not understood by a Windows terminal. If I would not know about cygwin, I would have issued the commands on each file manually, or would have written a batch file, probably. E.g.:
      pltotf esab8a.pl esab8a.tfm
      pltotf esab8r.pl esab8r.tfm
      ...
       
      vptovf esab7t.vpl esab7t.vf esab7t.tfm 
      vptovf esab8c.vpl esab8c.vf esab8c.tfm 
      ...  
       
      These files were generated with the pltotf command:

      esab8a.tfm 
      esari8a.tfm
      esari8r.tfm
      esaro8r.tfm
      esab8r.tfm 
      esabi8a.tfm
      esabi8r.tfm
      esabo8r.tfm
      esar8a.tfm 
      esar8r.tfm 

      And these files were generated with the vptovf command:

      esab7t.tfm  esab7t.vf  
      esab8c.tfm  esab8c.vf  
      esab8t.tfm  esab8t.vf  
      esabc7t.tfm esabc7t.vf 
      esabc8t.tfm esabc8t.vf 
      esabi7t.tfm esabi7t.vf 
      esabi8c.tfm esabi8c.vf 
      esabi8t.tfm esabi8t.vf 
      esabo7t.tfm esabo7t.vf 
      esabo8c.tfm esabo8c.vf 
      esabo8t.tfm esabo8t.vf 
      esar7t.tfm  esar7t.vf  
      esar8c.tfm  esar8c.vf  
      esar8t.tfm  esar8t.vf  
      esarc7t.tfm esarc7t.vf 
      esarc8t.tfm esarc8t.vf 
      esari7t.tfm esari7t.vf 
      esari8c.tfm esari8c.vf 
      esari8t.tfm esari8t.vf 
      esaro7t.tfm esaro7t.vf 
      esaro8c.tfm esaro8c.vf 
      esaro8t.tfm

      Testing the created fonts

      The answer of a question in the tex stackexchange group gave me some insides of the possibility to actually use the generated font support files without installing them to the MiKTex system.
      https://tex.stackexchange.com/questions/265840/texlive-installing-a-type-1-font

      Here is my test.ltx file that I used:
      \documentclass{letter}

      \begin{document}
      \fontfamily{esa}\fontseries{m}\fontshape{n}\selectfont
      Test sentence.
      \end{document}
      Creating the device independent (dvi) file with latex:
      >latex test.ltx


      Succeeded without error. Next converting the dvi into a postscript (ps) file.
      >dvips text.dvi

      The created pdf turned out to look like this:
      It turned out that the afm files were not properly generated by the TransType program.
      Note that I did the step of creating the pdf after installation of the map file.

      It seems not possible for the dvips program to map the font files without installing the map file.

       Correcting the Adobe font metrics files

      After discovering that the afm files generated by the TransType program are corruptes,  I manually corrected the files.
      In particular a semicolon of some lines were missing, e.g.:
      C -1 ; WX 605 ; N Aogonek ; B 15 -200 590 710 ;
      C -1 ; WX 280 ; N CR ; B 0 0 0 0
      C -1 ; WX 475 ; N Cacute ; B 60 -10 443 910 ;
      This were in total 4 occurances in each file. Changed the second line into e.g.:
      C -1 ; WX 280 ; N CR ; B 0 0 0 0 ;
      Invoking the tex interpreter on esa-drv.tex again, gave no errors this time (no need to press the return key several times). A lot of warnings were however still issued, like e.g.:
      Warning: missing glyph `Gamma'.
      Warning: missing glyph `Theta'.
      Warning: missing glyph `Lambda'.
      The generation of vf and tfm files with pltotf and vftopfissued still quite some "sorry's"
      Sorry, LIGTABLE too long for me to handle (line 10198).
         (KRN D 107 R -0.005) (COMMENT k)
      Sorry, LIGTABLE too long for me to handle (line 10199).
         (KRN D 108 R -0.005) (COMMENT l)
      And ended with:
      LIG character examined by '214 had no CHARACTER spec.
      Sorry, I haven't room for so many ligature/kern pairs!
      All ligatures will be cleared.
      I had to round some heights by 0000000 units.
      I had to round some depths by 0000000 units.

      Creating a map file

      The installfont script, in its next step creates a tex file that should create a map file that provides information for the dvi interpreter a mapping between fonts used in the TeX world and the binary fonts provided in extra files.
      Creatated the esa-map.tex file with following contents:
      \input finstmsc.sty
      \resetstr{PSfontsuffix}{.pfb}
      \adddriver{dvips}{esa.map}
      \input esa-rec.tex
      \donedrivers
      \bye
      Running tex on the file produxes the esa.map files:
      >tex esa-map.tex

      The contents of the esa.map file are:
      esar8r NotesEsa <[8r.enc <esar8a.pfb " TeXBase1Encoding ReEncodeFont "
      esaro8r NotesEsa <[8r.enc <esar8a.pfb " TeXBase1Encoding ReEncodeFont 0.167 SlantFont "
      esari8r NotesEsa-Italic <[8r.enc <esari8a.pfb " TeXBase1Encoding ReEncodeFont "
      esab8r NotesEsa-Bold <[8r.enc <esab8a.pfb " TeXBase1Encoding ReEncodeFont "
      esabo8r NotesEsa-Bold <[8r.enc <esab8a.pfb " TeXBase1Encoding ReEncodeFont 0.167 SlantFont "
      esabi8r NotesEsa-BoldItalic <[8r.enc <esabi8a.pfb " TeXBase1Encoding ReEncodeFont "

      So it does kind off associate a specific Adobe font binary (pfb) in "8a" (TeX 8-bit text (T1))  encoding with "8r" (TeX 8-bit 'raw' (TeXBase1)) encoding. 

      Creating a portable document format (pdf)

      Next steps are to create a PostScript (ps) file from the dvi file. And following a pdf from the ps file.
      Apparently the latex program used the mtx files to determine the space and position needed by each character.  The dvitops program then puts the actual characters at those reserved spots from the pfb files. The dvips program came up with some errors:

      ap is an unknown METAFONT mode.
      dvips: Font esar8r not found; characters will be left blank.
      This error appears to to the lack of the

      Installing the font and support files

      Some information on how to install, and also use, the generated font I used some of the folling information:
      https://tex.stackexchange.com/questions/88423/manual-font-installation

      As done by the installfont script itself and advices by the updmap.cfg file (in my case in C:\Program Files\MiKTeX 2.9\miktex\config) itself.
      initexmf  --edit-config-file updmap
      And added the line  the following line to the file:
      Map esa.map
      The program created with the previous command a new updmap.cfg file with the following contents:
      %% T1 font configuration settings. See the MiKTeX manual for help.
      in the directory:
      C:\Users\<nonAdminRightUser>\AppData\Roaming\MiKTeX\2.9\miktex\config
      For the dvips program to be able to use the map contents I needed to issue the following command:
      initexmf --mkmaps
      The above steps are sufficient to create a pdf with the new esa font. The initexmf --mkmaps command need to be issued from the directory in which the esa.map file is located.


      The initexmf --mkmaps command does issue the mkfntmap command and left its log at:
      C:\Users\<nonAdminRightUser>\AppData\Local\MiKTeX\2.9\miktex\log\mkfntmap.log
      2017-07-20 20:34:12,764+0200 INFO  mkfntmap - Parsing .\esa.map...
      So it was possible to check if the map file was actually picked up by MiKTeX.

      The often used font installation guide from Phillip Lehman was located at time of writing at:
      C:\Program Files\MiKTeX 2.9\doc\guides\fontinstallationguide\fontinstallationguide.pdf or internet
      http://ctan.triasinformatica.nl/info/Type1fonts/fontinstallationguide/fontinstallationguide.pdf

      In this guide it is said the map file should be copied to "dvips/config/ in the local TeX tree".
      But the mkfntmap program did not pick it up only after running initexmf --update-fndb. I tried the following dirs:
      C:\Users\<nonAdminRightUser>\AppData\Roaming\MiKTeX\2.9\fonts\map\dvips\esa\esa.map
      Also here it was successfully picked up:
      C:\localtexmf\fonts\map\dvips\esa\esa.map
      which definitely is my preferred path, since this is my private path, that is not in danger :) by the MiKTeX installation system. I set this previously up in the MiKTeX settings (admin) application.
      i shall note that I do have a multi-user Windows 10 system. While I am doing the font installation on my user, while I do not have administrator rights. I can however run e.g. the MiKTeX settings (admin) application under the administrator user which gives it elevated access to the rest of the system.
      This map installation or usage was terrible, I had no idea who was working because of what and whom. It was an endless copying around and trying. There are so many paths and user/access combination, that this took really long to figure out whats going on. Especially in the beginning when the TransType program produced unusable font files.

      The other files I copied according to the font installation guide chapter 1.4 "installing font and support files" to the appropriate directories.
      These were:
      tfm files to:
      C:\localtexmf\fonts\tfm\esa
      pfb files to:
      C:\localtexmf\fonts\type1\esa
      vf files to:
      C:\localtexmf\fonts\vf\esa
      fd files to:
      C:\localtexmf\tex\latex\esa

      Note: I created these folder myself (including C:\localtexmf). This I configured to be my local texmf root with the MiKTeX settings (admin) application.


      I guess I needed to run initexmf --update-fndb again here. I did run this command so often that I lost track and patience to remove the files again to try if this command actually made it happen that the font files were found by the appropriate programs.
      I created the final test pdf  from the following source file "animate.ltx":
      \documentclass[pstricks]{standalone}
      \usepackage{pstricks-add}

      \begin{document}
          \multido{\rA=-1.0+0.1}{20}
          {
              \begin{pspicture}(-2.0,-0.5)(2.0,0.5)
                  \psline(-2.0,0)(2.0,0)
                  \rput[B](\rA,0){\fontfamily{esa}\fontseries{m}\fontshape{n}\selectfont Test sentence.}
              \end{pspicture}
          }
      \end{document}





      With the following commands:
      >latex animate.ltx
      >dvips animate.dvi
      >ps2pdf animate.pdf

      This created a pdf with 20 tiny pages. This pages I created with the convert program that was contained as a legacy tool in the imagemagic toolset. I did this only to be able to upload it here.
      >convert -delay 10 -density 600 -alpha off animate.pdf animate.gif

      Yeah. Such a large step for a human. And no impact for men at all ;)








      Saturday, March 23, 2013

      Why do not simply use QtGStreamer?

      Off course the question arises: "Why do not simply use QtGStreamer?" Ya, actually this should be the way to do. Since redoing it by your self is often useless. The downside is: currently only gstreamer-0.10 is supported and on updates this will not change. And the installation on Windows is difficult.
      In QtGstreamer there are principally two ways of integrating a video into a GUI:
      1) QGst::Ui::GraphicsVideoWidget (in conjunction with QGst::Ui::GraphicsVideoSurface)
      2) QGst::Ui::VideoWidget

      Here is an example for option 1:

      qtgraphicswidget.pro


      main.cpp:



      And here is an example for option 2:

      main.cpp:


      The project file can be the sane as for option 1

      Thursday, March 14, 2013

      Xoverlay

      Last time I wanted to fix some issues with the conversion from a gst_buffer video frame into a QImage. I fixed one of three issues. But before fixing the other two:
      • The mutex might be not optimally designed since it's a producer newSample() and a consumer getFrame()
      • A serious problem is that the data of the QImage must remain valid also if the image is used from the Display class. This is not given here since the lifetime of the GstBuffer is unknown
      I will try another approach: To use the xvimagesink directly. Here is the working source code:

      xoverlay.pro


      For the video player I will use a gst_video_overlay element which is part of the gstreamer base plugins (http://gstreamer.freedesktop.org/data/doc/gstreamer/head/gst-plugins-base-libs/html/gst-plugins-base-libs-gstvideooverlay.html).
      Nevertheless pkg-config gave following insight:
      shows:
      -pthread -lgstreamer-1.0 -lgobject-2.0 -lgmodule-2.0 -lgthread-2.0 -lglib-2.0
      shows:
      -pthread -lgstvideo-1.0 -lgstbase-1.0 -lgstreamer-1.0 -lgobject-2.0 -lgmodule-2.0 -lgthread-2.0 -lglib-2.0

      Thus it seems they mixed some things up and in order to use gst_video_overlay one has to include gstreamer-video-1.0 instead of gstreamer-plugins-base-1.0

      main.cpp


      videoplayer.h


      videoplayer.cpp


      I remember I got that idea from somewhere in the Internet, but i don't remember where. But it looks easy to use. This does however not work in Windows and I will avoid that solution in the future.
      Note that the size of the widget is hard coded to 320x240. This results from the default caps negotiation between the videotestsrc and the xvimagesink.

      Monday, March 11, 2013

      Last time I was showing three problems with the conversion from a gst buffer data to QImage.
      The code snipped was:

      And the problems were:
      • The mutex might be not optimally designed since it's a producer newSample() and a consumer getFrame()
      • The QImage construction should be done with const data since the data is not intended to get changed
      • A serious problem is that the data of the QImage must remain valid also if the image is used from the Display class. This is not given here since the lifetime of the GstBuffer is unknown

      I was trying to solve the simplest first: construct for const data. The difference as described in the Qt documentation is
      Unlike the similar QImage constructor that takes a non-const data buffer, this version will never alter the contents of the buffer. For example, calling QImage::bits() will return a deep copy of the image, rather than the buffer passed to the constructor. This allows for the efficiency of constructing a QImage from raw data, without the possibility of the raw data being changed.

      For a solution I intended to use the const_cast operator. And it seems it is working fine:


      The other two are somewhat more complex to solve. I will continue with the lifetime of the gstBuffer.

      Saturday, March 9, 2013

      Displaying a Videotestsrc in a Qt Application

      Since quite a while I am trying to get gstreamer working properly inside of Qt applications. Whereas I am not taking sound into account, solely video. I wanted to use gstreamer-1.0 and since the QtGstreamer binding is only binding gstreamer-0.10 this was not possible to use. Honestly speaking QtGstreamer with the older gstreamer might have been the better choice. But I also wanted to know how it works and switch later to the Qt binding. There is also hope that the developer will port the Qt binding to gstreamer-1.0. My first testing application I wrote with QtCreator. Here is the source code:


      videotest.pro



      main.cpp



      capture.h


      capture.cpp

      The Capture class is a subclass of QLabel only because I intended to include a signal which is emitted each time a frame arrives. The approach is based on the gst_app_sink plugin. With the caps "video/x-raw, format=RGB, bpp=8" and the correct dimension the data can be safely mapped to a QImage with QImage::Format_RGB888 format. The whole code of getting the data from the gstBuffer in newSample() is very different from the approach in gstreamer-0.10. I find the way of gstreamer-1.0 pretty tedious. Several problems remain here:
      • The mutex might be not optimally designed since its a producer newSample() and a consumer getFrame()
      • The QImage construction should be done with const data since the data is not intended to get changed
      • A serious problem is that the data of the QImage must remain valid also if the image is used from the Display class. This is not given here since the lifetime of the GstBuffer is unknown


      display.h


      display.cpp


      The Display class is a subclassed QLabel. For the testing purpose it is sufficient to display the QLabel widget only. In a real applications this widget can be included in a QMainWindow or QDialog easily. The paintEvent is overwritten to enable the rendering of the pixmap. I don't know why the paintEvent occurs frequently enough without doing nothing. There is no explicit update() or repaint() slot executed.

      Its raining ...

      A test for the syntax highlighter:
      I followed the advices at Geek Talkin Siebel: Embed Code Syntax Highlighting in Blog. Thus I copied the the source code into this blog's template HTML code. After putting it in the compose mode the source was not formated. After adding the source code directly into the HTML edit mode, the source code was properly highlighted. Unfortunately the whole source is not visible in the "Compose" mode.
      It is however not straightforward to cascade the code in order to show the source of the above embedded code. With the <pre> it was however possible:
      
      
      And the above with:
      
      
      The above code was generated with another <pre> node.