/[volute]/trunk/projects/ivoapub/ivoatexDoc/ivoatexDoc.tex
ViewVC logotype

Contents of /trunk/projects/ivoapub/ivoatexDoc/ivoatexDoc.tex

Parent Directory Parent Directory | Revision Log Revision Log


Revision 3171 - (show annotations)
Thu Dec 3 07:04:05 2015 UTC (4 years, 9 months ago) by msdemlei
File MIME type: application/x-tex
File size: 40015 byte(s)
ivoatexDoc: mentioning cmd_xxx facility in update_generated.


1 \documentclass[11pt,a4paper]{ivoa}
2 \input tthdefs
3
4 \usepackage{listings}
5 \lstloadlanguages{sh,make,[latex]tex}
6 \lstset{flexiblecolumns=true,numberstyle=\small,showstringspaces=False,
7 identifierstyle=\texttt,defaultdialect=[latex]tex,language=tex}
8
9 \usepackage{todonotes}
10
11 \usepackage[utf8]{inputenc}
12
13 \definecolor{texcolor}{rgb}{0.4,0.1,0.1}
14 \newcommand{\texword}[1]{\texttt{\color{texcolor} #1}}
15
16 \iftth
17 \newcommand{\BibTeX}{BibTeX}
18 \fi
19
20 \iftth
21 \newcommand{\comicstuff}[1]{
22 \begin{html}<span class="comic">#1</span>\end{html}}
23 \else
24 \newcommand{\comicstuff}[1]{(HTML exclusive material)}
25 \fi
26 \customcss{custom.css}
27
28 \title{The \ivoatex\ Document Preparation System}
29
30 \ivoagroup{Standards and Processes}
31
32 \author[http://www.ivoa.net/cgi-bin/twiki/bin/view/IVOA/MarkusDemleitner]{Markus Demleitner}
33 \author[http://www.ivoa.net/cgi-bin/twiki/bin/view/IVOA/MarkTaylor]{Mark Taylor}
34 \author[http://www.ivoa.net/cgi-bin/twiki/bin/view/IVOA/PaulHarrison]{Paul Harrison}
35 \author[http://www.ivoa.net/cgi-bin/twiki/bin/view/IVOA/MarcoMolinaro]{Marco Molinaro}
36
37 \editor{Markus Demleitner}
38
39 \previousversion[http://www.ivoa.net/documents/ivoatexDoc/20150129][Version
40 1.0]
41
42
43 \begin{document}
44
45 \SVN$Rev$
46 \SVN$Date$
47 \SVN$URL$
48
49 \begin{abstract}
50 This note describes the \ivoatex\ document preparation system for IVOA
51 standards and notes. \ivoatex\ supports the production of
52 PDF and HTML renderings of the documents with sources in
53 plain text suitable for version control, as is desirable for normative
54 texts. This note contains a user guide as well as a discussion of
55 \ivoatex's dependencies and its implementation.
56 \end{abstract}
57
58
59 \section*{Acknowledgments}
60
61 \ivoatex\ heavily draws from experiences made with previous markup-based
62 document preparation systems, in particular LaTeX classes and
63 infrastructure created by Sebasti\'en Derriere and Mark Taylor, as well
64 as Paul Harrison's XML-based ivoadoc system.
65
66 We thank tth's author, Ian Hutchinson, for generous technical support
67 and prompt provision of solutions in the upstream source where necessary.
68
69 \section{Introduction}
70
71 Creating and developing standards is a big part of the operations of
72 the International Virtual Observatory Association (IVOA).
73 As these are normative texts, attention to detail is very important, and
74 being able to rigorously track changes to the documents is highly
75 advantageous.
76
77 Standards are also often developed cooperatively, which means that
78 capabilities for branching and merging are desirable. This strongly
79 suggests employing version control systems for document authoring.
80 Change tracking in software designed for editing office documents, to
81 the extent it is supported at all, usually requires significant
82 manual intervention, is optional, often used incorrectly, and frequently
83 lacks interoperability. Led by these considerations, it was decided that
84 \ivoatex\ would have to be based on plain text source files.
85
86 As mandated by \citet{std:docSTD}, finished documents have to be at
87 least available in PDF, while an additional HTML rendering for online
88 use is recommended. A document preparation system should thus be
89 able to produce documents in these formats in at least acceptable
90 quality.
91
92 With these constraints in mind, several possible solutions were
93 investigated. Paul
94 Harrison's ivoadoc
95 system\footnote{\url{https://volute.g-vo.org/svn/trunk/projects/ivoapub/ivoadoc}}
96 went for XHTML as
97 an input format and used XSLT2 and XML-FO as document processors. While
98 this facilitated several interesting features -- for instance,
99 automatic extraction and formatting of XML schema fragments or
100 straightvorward embedding of RDFa markup for machine-readable examples
101 --, it turned out that tooling issues were severe (e.g., reliable use
102 of SGML catalogs\footnote{This is important as retrieval of DTDs and
103 similar data from their commonly used system identifiers (i.e.,
104 typically W3C web servers) is at least undesirable and in practice
105 causes massive delays in formatting due to rate limiting on the part of
106 the W3C.}, non-free hyphenation patterns, classpath issues) and the use
107 of XML-FO for PDF generation yielded inferior renderings with little
108 prospect for improvements by third parties. Also, authors disliked
109 writing HTML tags.
110
111 Other options considered for source languages included docbook or one of
112 the newer lightweight markup languages (ReStructuredText, markdown,
113 etc). In each case, there were concerns either regarding the system's power
114 and flexibility or its ease of installation and maintenance.
115
116 Meanwhile, several documents -- to mention just a few, SAMP, VOTable,
117 and VOUnits -- had successfully used \TeX-based systems typically
118 derived from work done in the early 2000s by Sebasti\'en Derriere.
119 \ivoatex\ essentially is a generalisation of these standards' formatting
120 systems, also inheriting from them the use of the \texttt{make} tool to
121 automate workflows.
122
123 \ivoatex\ was extended to relieve document editors from some of the
124 bookkeeping involved with producing IVOA standards and provide authors
125 with uniform solutions for common problems in standards typesetting.
126
127 In the remainder of this document, we give quick-start instructions
128 on installation and authoring in sect.~\ref{sect:quick} and continue
129 a more thorough discussion of \ivoatex's facilities with a special focus
130 on enabling proper automatic production of both HTML and PDF output in
131 sect.~\ref{sect:authoring}. In sect.~\ref{sect:impl}, additional
132 details on the implemenation are given for the benefit of authors
133 planning to extend \ivoatex. We close with a discussion of open issues
134 and desirable developments.
135
136 \section{Installation and Quick Start}
137 \label{sect:quick}
138
139 \subsection{Dependencies}
140
141 \ivoatex\ is designed to work more or less out of the box on common
142 POSIX-compliant systems; no non-free software is required for operation.
143 Its main uncommon dependency is the tth translator, which is a relatively
144 compact program
145 written in highly portable C generated through lex used to translate LaTeX
146 to HTML. As it is compact and portable, it is delivered with
147 \ivoatex~and built on demand.
148 As \ivoatex's tth may at times offer
149 some enhancements over the upstream tth, using a system-installed tth is
150 discouraged.
151
152 The remaining dependencies include:
153
154 \begin{itemize}
155 \item A \LaTeX\ distribution with some commonly available packages (calc,
156 graphicx, xcolor, ifthen, doc, paralist, url, natbib, caption, hyperref,
157 the agsm bibliography style). It is recommended to install TeXLive.
158 \item A sufficiently capable implementation of \texttt{make}, with GNU's
159 implementation recommended.
160 \item The XSLT1 processor \texttt{xsltproc} (a different processor can
161 be used, but that would probably require custom make rules).
162 \item The \texttt{gcc} compiler (another C compiler could be used; the
163 central makefile should probably be amended to allow easier changes
164 here).
165 \item The \texttt{zip} archiver for package generation.
166 \item \texttt{imageMagick} and \texttt{ghostscript} if vector graphics
167 is to be processed.
168 \end{itemize}
169
170 On Windows, it is recommended to run \ivoatex\ within cygwin, where all
171 dependencies can easily be installed from cygwin's repository.
172
173 On
174 Debian-derived systems, the dependencies should be present after
175 running a distribution-specific adaption of
176 \begin{lstlisting}[language=sh,basicstyle=\footnotesize]
177 apt-get install build-essential texlive-latex-extra zip xsltproc\
178 texlive-bibtex-extra imagemagick ghostscript cm-super
179 \end{lstlisting}
180 (cm-super contains vector versions of computer modern fonts in T1
181 encoding), on RPM-based systems something like
182 \begin{lstlisting}[language=sh]
183 yum install texlive-scheme-full libxslt make gcc zip\
184 ImageMagick ghostscript
185 \end{lstlisting}
186 should pull in everything that is necessary.
187
188 With OS X, a convenient way to obtain the dependencies is to install
189 MacPorts\footnote{\url{https://www.macports.org/}} and then run
190 \begin{lstlisting}[language=sh]
191 port install ImageMagick libxslt ghostscript +full
192 \end{lstlisting}
193 The canonical OS\,X \TeX\ distribution is the Mac\TeX\ version of
194 \TeX Live\footnote{\url{https://www.tug.org/mactex/}}. It is also
195 possible to build \TeX\ using MacPorts (with \texttt{port install
196 texlive}), but this may result in a slightly non-standard
197 distribution.\footnote{See
198 \url{http://tex.stackexchange.com/questions/97183/}.
199 Also, MacPorts does add
200 \texttt{texlive} as a dependency on many packages, and so frequently
201 \emph{insists} on trying to build it; if you want to prevent this,
202 there is some discussion at
203 \url{http://comments.gmane.org/gmane.os.apple.macports.user/21526}.}
204
205 To see if the prerequisites are there and compatible with \ivoatex, try
206 building an updated version of this document from its volute source:
207 \begin{lstlisting}[language=sh]
208 svn co https://volute.g-vo.org/svn/trunk/projects/ivoapub/ivoatexDoc
209 cd ivoatexDoc
210 make biblio # update the bibliography
211 make forcetex # make a PDF ignoring timestamps
212 make ivoatexDoc.html # make an html document
213 make package # make a zipfile for IVOA submission
214 \end{lstlisting}
215 Various diagnostics both from tth and from xsltproc (unknown commands,
216 unexpected end tags, and the like) are
217 expected at this point and no reason for alarm; we work on reducing the
218 amount of false alarm error messages.
219
220 \subsection{Quick Start}
221
222 For ease of installation and robustness, \ivoatex\ for now is designed
223 to be used from within a subdirectory of the directory containing the
224 document sources (rather than being installed globally). Given that
225 it is fairly compact, having one copy per document seems acceptable.
226
227 So, the first step to use ivoatex is to create a development
228 directory:
229
230 \begin{lstlisting}[language=sh]
231 export DOCNAME=SampleDoc
232 # this would be your document's short name, e.g., RegTAP, SIAv2)
233 mkdir $DOCNAME
234 \end{lstlisting}
235
236 The DOCNAME -- which will turn up in URLs, standard identifiers, and the
237 like -- should be chosen to be both succinct and expressive, and it
238 should not contain non-alphanumeric characters (the examples given here
239 assume that, too). A name
240 like \texttt{SimpleDALRegExt} probably marks the upper limit in terms of
241 length.
242
243 While it is clearly preferable if authors use IVOA's
244 designated common version control
245 system -- at this point, this is
246 Volute\footnote{http://volute.g-vo.org} -- from the outset of
247 document development, it is possible to operate it locally as well.
248
249 \subsubsection{Installation from Archive (without Volute)}
250
251 Without version control, it is sufficient to obtain \ivoatex\ from a
252 distribution site and unpack it into the future document directory:
253
254 \begin{lstlisting}[language=sh]
255 cd $DOCNAME
256 curl http://soft.g-vo.org/ivoatex/ivoatex-latest.tar.gz \
257 | tar -xvzf -k
258 \end{lstlisting}
259
260 \subsubsection{Installation with SVN version control}
261
262 While it is of course possible to keep \ivoatex\ in checkouts, too,
263 the recommended and more elegant way is to use \texttt{svn:externals}.
264
265 \begin{lstlisting}[language=sh,basicstyle=\footnotesize]
266 export VOLUTEBASE="https://volute.g-vo.org/svn/trunk/projects"
267 export WG=?????
268 # this would be the Working Group name as represented in
269 # volute subdirectories: dal, dm, edu, grid, registry
270 svn import $DOCNAME $VOLUTEBASE/$WG/$DOCNAME
271 rm -r $DOCNAME
272 svn co $VOLUTEBASE/$WG/$DOCNAME $DOCNAME
273 cd $DOCNAME
274 svn propset svn:externals\
275 "ivoatex $VOLUTEBASE/ivoapub/ivoatex" .
276 svn update
277 svn propset svn:ignore . --file ivoatex/svn-ignore.txt
278 \end{lstlisting}
279
280 This has the advantage that \ivoatex~updates are automatically pulled in
281 and that it is
282 trivial to feed back bibliography additions and patches to \ivoatex\
283 itself.
284
285 \subsubsection{Beginning the document}
286 \label{sect:beginning}
287
288 For convenience, the document production should start from some common
289 templates which are part of the \ivoatex\ distribution:
290
291 \begin{lstlisting}[language=sh]
292 cp ivoatex/Makefile.template Makefile
293 cp ivoatex/document.template $DOCNAME.tex
294 cp ivoatex/archdiag.png .
295 \end{lstlisting}
296
297 The next step is to fill out the makefile template.
298 In \ivoatex\ version 0.4, this template looks like this:
299
300 \begin{lstlisting}[language=make,basicstyle=\footnotesize]
301 # ivoatex Makefile. The ivoatex/README for the targets available.
302
303 # short name of your document (e.g., RegTAP)
304 DOCNAME = ????
305
306 # count up; you probably do not want to bother with versions <1.0
307 DOCVERSION = 1.0
308
309 # Publication date, ISO format; update manually for "releases"
310 DOCDATE = ???
311
312 # What is it you're writing: NOTE, WD, PR, or REC
313 DOCTYPE = ???
314
315 # Source files for the TeX document (but the main file must always
316 # be called $(DOCNAME).tex
317 SOURCES = $(DOCNAME).tex
318
319 # List of pixel image files to be included in submitted package
320 FIGURES = archdiag.png
321
322 # List of PDF figures (for vector graphics)
323 VECTORFIGURES =
324
325 include ivoatex/Makefile
326 \end{lstlisting}
327
328 All lines with question marks must be filled out. The document date
329 is the publication date, which can be significantly different from the
330 current date. After its initial setting, it should only be changed at
331 the time of submission to the document archive. It is always in
332 DALI-style ISO format \citep{std:DALI}, e.g., 2014-03-31.
333
334 \texttt{SOURCES} is used in dependency processing. It would be amended
335 when the source file is split into separate files or if material is
336 included into the document, e.g., via \texword{lstinputlisting}.
337 Graphics files included do not need to be given here.
338
339 \texttt{FIGURES} must contain the names of all bitmap graphics included
340 in the document; files missing here will be missing from the package for
341 distribution to the IVOA document repository, which will break the HTML
342 rendering. As to \texttt{VECTORFIGURES}, see
343 sect.~\ref{sect:vectorgraphics}.
344
345 The template for the \TeX\ source contains several lines with
346 multiple question marks. These must be filled out as well.
347
348 As illustrated in the template, both \texword{author} and
349 \texword{previousversion} support an optional argument giving an URL; for
350 \texword{author}, it should normally point to the respective person's
351 page in the IVOA wiki, for \texword{previousversion}, it should point to
352 the landing page of the respective document version in the IVOA document
353 repository. Further automation for maintaining document history
354 certainly is desirable, and the authors welcome ideas for how this might
355 look.
356
357 The architecture diagram specimen \texttt{archdiag.png} is only
358 necessary for documents on the recommendation track (i.e., notes in
359 general do not have one). Architecture diagrams adapted to the concrete
360 document are currently prepared by the chair of the Technical
361 Coordination Group on request. If no architecture diagram is present
362 within a document, the reference to it from \texttt{FIGURES} in the
363 makefile must be removed.
364
365 \ivoatex requires the source to be written in UTF-8 encoding, since the
366 references shipped with it are in UTF-8.
367 Authors are urged to keep lines shorter than 72 characters in input
368 files whenever possible in order to keep diffs useful and readable.
369 When edits are made, paragraphs should not normally be reflowed to avoid
370 large diffs for minor edits.
371
372 The PDF version of the document is built by the makefile's default rule,
373 so running \texttt{make} will usually by enough. This will produce a
374 file \texttt{\$DOCNAME.pdf}.
375
376 Other makefile targets for author use include:
377
378 \begin{itemize}
379 \item \texttt{biblio} updates the bibliography (i.e., runs \BibTeX);
380 running this is necessary after one of the bibliography files is updated
381 or when a new publication is referenced from the document.
382 \item \texttt{forecetex} rebuilds the PDF unconditionally (e.g., when TeX
383 asks to be rerun)
384 \item \texttt{\$DOCNAME.html} generates an HTML rendering of the
385 document; at this point, this will typically emit quite a bit of
386 spurious diagnostics. Unfortunately, real problems may hide within. In
387 the end, we currently recommend a visual inspection of the resulting
388 HTML before submission.
389 \item \texttt{package} generates a zip file containing everything needed
390 for publication in the IVOA's document repository. Obviously,
391 \texttt{DOCVERSION}, \texttt{DOCTYPE}, and \texttt{DOCDATE} in the
392 Makefile should be updated as necessary before this target is built.
393 The result is a zip file with a name compliant to the IVOA document
394 standards \citep{std:docSTD}.
395 \item \texttt{generate} is used to update machine-generated content in
396 the document, typically by the main editor. See
397 section~\ref{sect:generated} for details.
398 \end{itemize}
399
400 \begin{admonition}{Note}
401 Simply running \texttt{latex} or \texttt{pdflatex} directly
402 (rather than through make) is \emph{not} supported with \ivoatex. Due
403 to the non-global installation of the support files, the \TeX\ run needs
404 a special environment that is prepared by the makefile.
405
406 Also note that bibliography processing must be initated manually by
407 running \texttt{make biblio}; unless authors checked in the bbl file
408 this produces, this must be run after a document checkout.
409 \end{admonition}
410
411 \subsection{Examples}
412
413 Examples for \ivoatex\ use include this
414 document\footnote{https://volute.g-vo.org/svn/trunk/projects/ivoapub/ivoatexDoc}
415 or
416 RegTAP\footnote{https://volute.g-vo.org/svn/trunk/projects/registry/regtap}.
417 At the time of writing, about half a dozen IVOA documents within Volute
418 are based on \ivoatex.
419
420
421 \section{Authoring documents}
422 \label{sect:authoring}
423
424 While \ivoatex\ documents can be written much like any other \TeX\
425 document, it is advisable to follow certain standards and use special
426 facilities for common appearance, easier development, and possible
427 evolution of \ivoatex\ itself.
428
429 \subsection{\ivoatex\ Features}
430
431 \ivoatex\ provides a small set of macros and environments designed
432 to ease standards authoring. These include:
433
434 \begin{bigdescription}
435 \item[\texword{author}, \texword{previousversion}] these are discussed
436 in sect.~\ref{sect:beginning}.
437 \item[\texword{xmlel}] a macro for marking up XML element or attribute
438 names and similar.
439 \item[\texword{vorent}] for a name taken from VOResource, usually an
440 element or attribute name.
441 \item[\texword{admonition}] This is an environment for
442 displayed boxes, intended for notes, tips, and the like.
443 It takes an argument giving the head of the box, e.g.,
444
445 \begin{lstlisting}[language=TeX]
446 \begin{admonition}{Note}
447 Admonitions should not be overdone.
448 Also, they are floating insertions.
449 \end{admonition}
450 \end{lstlisting}
451 \item[\texword{bigdescription}] This is an environment for definition
452 lists in the style of HTML \xmlel{dl}, and it will be translated into
453 one. Use \verb|\item[term]| for the term to be defined
454 (the construct this item is in is a \texword{bigdescription}).
455 \end{bigdescription}
456
457 The intention behind macros like \texword{xmlel} and \texword{vorent} is
458 that such terms are typeset uniformly across documents. Further
459 semantic markup like this is planned for future releases, and document
460 authors are encouraged to contribute terms.
461
462 Also note that the title page is generated by the abstract environment.
463 Thus, all \ivoatex\ documents must have an abstract within the
464 \texword{abstract} environment.
465
466 \subsection{Listings, Verbatim Material}
467
468 \ivoatex\ documents should use the \texword{listings} package to include
469 source code snippets, XML fragments and the like. It can be used after
470 a set of declarations like
471 \begin{lstlisting}[language=TeX]
472 \usepackage{listings}
473 \lstloadlanguages{XML,sh}
474 \lstset{flexiblecolumns=true,tagstyle=\ttfamily,
475 showstringspaces=False}
476 \end{lstlisting}
477 is included in the document preamble (i.e., before its
478 \verb|\begin{document}|). The languages preloaded should be adapted to
479 the document's needs. Additional
480 languages supported that are likely relevant in a VO context include C,
481 fortran, python, SQL, and java, specified as above case-insensitively.
482
483 The setup in the example (flexible columns, tags in
484 typewriter, no explits blanks even in strings)
485 is recommended for including XML source.
486
487 Actual listings are obtained with code like
488 \begin{verbatim}
489 \begin{lstlisting}[language=XML]
490 <example id="empty"/>
491 \end{lstlisting}
492 \end{verbatim}
493 Alternatively, entire files can be included like this:
494 \begin{verbatim}
495 \lstinputlisting[language=XSLT]{makeutypes.xslt}
496 \end{verbatim}
497 In the PDF rendering, the listings are pretty-printed. In the HTML
498 rendering, the content is, currently, simply included in \xmlel{pre} elements.
499
500 If a more compact rendering of listings is desired, for instance,
501 because larger portions of source code are required in the document,
502 listings' \texword{basicstyle} option should be used together with one
503 of LaTeX standard size macros. This could be in the argument of
504 \texword{lstset} in the preamble for a global setting, or on a
505 case-by-case basis as in
506
507 \lstinputlisting[language=tex]{verbatimstyles.tex}
508
509 As of version 0.5 of \ivoatex, only \texword{footnotesize} is actually
510 formatted by the CSS embedded in the HTML document, and we believe
511 listings should not be much smaller than that anyway. As the options
512 are translated into CSS classes, it is fairly easy to add further
513 formatting functionality on a document-by-document basis, though.
514
515 \subsection{References and Bibliography}
516
517 \ivoatex\ documents should use natbib and \BibTeX\ to manage references.
518 The package comes with a default bibliography in
519 \texttt{ivoatex/ivoabib.bib} containing records for many publications
520 likely to be cited by IVOA documents. An additional bibliograhy of
521 IVOA documents present in the document repository will be added soon.
522
523 As usual in natbib, actual references are made through either writing
524 \verb|\citep{tag}|, yielding a form like ``(Einstein 1905)'',
525 or \verb|\citet{tag}|, yielding a form like ``Einstein (1905)''.
526 \ivoatex\ does not support variant forms of citep and citet (i.e., those
527 with optional arguments) yet; they will work in PDF output but fail in
528 HTML. Contributions to improve this are welcome.
529
530
531 If a source does not already have a record in the bibliography that
532 comes with \ivoatex, authors are welcome to contribute new records into
533 its main repository. Authors expecting that a reference really will not
534 be reused in other documents can have a local bibliography as well.
535 They will then need to amend the bibliography definition near the end of
536 the source file to, for instance,
537 \begin{lstlisting}[language=tex]
538 \bibliography{ivoatex/ivoabib,local}
539 \end{lstlisting}
540 (assuming the local records are stored in \texttt{local.bib}).
541
542 Changes that introduce new references or additions to the bibliography
543 require that \texttt{make biblio} is run afterwards.
544
545 \subsection{Graphics}
546
547 \subsubsection{Bitmap Graphics}
548
549 \ivoatex\ supports all bitmap graphics formats that pdflatex supports.
550 In practice, authors are encouraged to restrict themselves to JPEG, PNG,
551 and possibly GIF. Currently, identical images are used for both PDF and
552 HTML renderings. The recommended pattern for figures is
553 \begin{lstlisting}[language=tex]
554 \begin{figure}[thm]
555 \begin{center}
556 \includegraphics[width=0.9\textwidth]{mydiagram.png}
557 \end{center}
558 \caption{A diagram of what this is about.}
559 \label{fig:mydiag}
560 \end{figure}
561 \end{lstlisting}
562 This gives \LaTeX\ some leeway in placing the figure, defines the image
563 size in units of the page width, and centers the image itself.
564
565 All bitmap graphics in a document must be listed in the makefile's
566 \texttt{FIGURES} variable. If they are not, the HTML rendering will be
567 broken.
568
569 \subsubsection{Vector Graphics}
570 \label{sect:vectorgraphics}
571
572 The only vector graphics format supported in \ivoatex\ is PDF. PDF
573 files can be directly used in \texword{includegraphics}. The names of
574 such figures must be listed in the makefile's \texttt{VECTORFIGURES}
575 variable.
576
577 From \texttt{VECTORFIGURES}, \ivoatex\ arranges that, when a PDF figure
578 \texttt{foo.pdf} is used, the HTML target depends on a file called
579 \texttt{foo.png}. This PNG can be generated automatically by
580 \ivoatex\ using a combination of ghostscript and ImageMagick. It may
581 sometimes be preferable to perform a custom conversion by hand (e.g.,
582 more compact representation with bilevel source images), in which case
583 the pre-rendered PNG should be included in the version controled
584 repository. This also
585 has the advantage that neither ghostscript nor ImageMagick are build
586 dependencies of the document.
587
588 \subsection{Tables}
589
590 In tables, rules should be used sparingly. The standard pattern for tables is
591 something like
592 \begin{lstlisting}
593 \begin{table}[thm]
594 \begin{tabular}{p{0.35\textwidth}p{0.64\textwidth}}
595 \sptablerule
596 \textbf{Column Head}&\textbf{Another column head}\\
597 \sptablerule
598 A value & Another value\\
599 A value in row 2& And so on\\
600 \sptablerule
601 \caption{A sample table}
602 \label{table:extable}
603 \end{tabular}
604 \end{table}
605 \end{lstlisting}
606
607 The \texword{sptablerule} used here inserts a horizontal rule with some
608 extra spacing and will be rendered consistently in both PDF and HTML.
609 It should not, as a rule, used between table rows, it is intended
610 primarily to delimit the table itself as well as the the heading and the
611 body.
612
613 \subsection{Hyperlinks}
614
615 While \ivoatex\ puts no restrictions on the usage of hyperref features,
616 the preferred way to include links in \ivoatex\ documents is to use the
617 \texword{url} macro, i.e., use the URL itself as the anchor text. In
618 this way, the link remains (to some extent) usable even if the document
619 is printed. The alternative two-argument \texword{href} should
620 generally be avoided as it fails on paper. For instance,
621 \begin{lstlisting}
622 (this is bad:) The \href{http://ivoa.net}{IVOA} has issued
623 \href{http://ivoa.net/documents}{many standards}.
624 \end{lstlisting}
625 would severely degrade when printed and is hence discouraged, whereas
626 \begin{lstlisting}
627 The IVOA\footnote{\url{http://ivoa.net}} has issued many
628 standards, all of which can be retrieved from
629 \url{http://ivoa.net/documents}.
630 \end{lstlisting}
631 works properly on all of \ivoatex's target media.
632
633 With non-HTTP URIs, it is recommended to use hyperref's
634 \texword{nolinkurl} macro (rather than an unadorned \texword{textt} or
635 similar); advantages include that line breaking is better with
636 nolinkurl, less manual escaping is necessary, and, if desired,
637 such URIs can be more easily styled. An example:
638
639 \begin{lstlisting}
640 The value of the \xmlel{standardID} attribute will be
641 be \nolinkurl{ivo://ivoa.net/std/Registry#OAI-2.0}.
642 \end{lstlisting}
643
644 \subsection{Editorial tools}
645
646 When authoring standards, it is sometimes necessary to include
647 editorial comments of the type ``Need to clarify'' or ``Specification
648 incomplete''. We recommend to use the todonotes package for such
649 pieces of text\footnote{Full documentation is available at
650 \url{http://www.tex.ac.uk/CTAN/macros/latex/contrib/todonotes/todonotes.pdf}.}.
651 The recommended usage is like
652 \begin{lstlisting}
653 ...
654 \usepackage{todonotes}
655 ...
656 \begin{document}
657 \todo{This is an example for a editorial note}.
658 \end{lstlisting}
659
660 A rendering of such a to-do note is shown in this paragraph
661 \todo{This is an example for a editorial note}. In HTML output,
662 only the simple \texword{todo} macro without options is supported, and
663 text is simply displayed inline right now. Note in particular that
664 todonote's \texword{obeyDraft} and \texword{obeyFinal} package options
665 are ignored. Although \ivoatex\ does not enforce it (yet), finished
666 recommendations should have no todo items in them.
667
668 While todonotes is a useful tool for standards development, we
669 discourage the use of packages to mark up changes, as maintaining such
670 markup usually is very hard, and version control offers a more
671 manageable solution to the problem such packages attempt to solve.
672
673 \subsection{Version Control System Information}
674
675 It is recommended to include basic metadata obtained from the version
676 control system into \ivoatex~Documents where available. Basic support
677 for Volute (i.e., subversion) is built into \ivoatex. It is based on
678 subversion's
679 keyword substitution, which therefore needs to be enabled on the
680 main document source file.
681 While any keys might be used, \ivoatex's default support deals with:
682
683 \begin{lstlisting}[language=sh]
684 svn propset svn:keywords "Date Rev URL" $DOCNAME.tex
685 \end{lstlisting}
686
687 (only Rev is mandatory).
688 In the document itself, the text for subversion's replacement needs to
689 be included. Inititally, this must look like this:
690
691 \lstinputlisting[language=tex]{svnsubstitution.tex}
692
693 Subversion will, at every commit, enter the current values in a
694 controlled fashion, which will then be picked up by \ivoatex's macros.
695 Due to the way this information is processed by \TeX, after adding the above
696 text, a commit must be made before the document can be built again.
697
698 For non-English locales, subversion's substitution poses the problem that
699 localized information will be entered into the document, which is
700 unwelcome for many resaons (non-ASCII characters, ambiguous date
701 formats). We therefore recommend to run SVN in the C
702 locale, at least while working on IVOA documents. Users affected with
703 on-C locales
704 could in a Bourne-like shell use a construct like
705
706 \begin{lstlisting}[language=sh]
707 alias svn='LC_ALL=C svn'
708 \end{lstlisting}
709
710 and forget about the problem.
711
712
713 \subsection{Generated Content}
714 \label{sect:generated}
715
716 Sometimes it is desirable to have parts of a document generated through
717 some sort of ivoatex-external process. Examples include copying
718 documentation from XML Schema files into the standard document (used,
719 e.g., in TAPRegExt 1.1) or obtaining column metadata for standard
720 data models from a live TAP\_SCHEMA (used, e.g., in RegTAP 1.0).
721
722 For such cases, \ivoatex\ offers a python script
723 \texttt{update\_generated.py}, which is executed by the
724 \texttt{generate} make target. It simply looks for structured comments
725 in the main document and replaces what is between them with generated
726 contents. The opening line consists of a \TeX comment introducer, a
727 blank, the literal \texttt{GENERATED:}, another blank, and a command
728 line. The closing line is a comment consisting of \texttt{/GENERATED}.
729
730 On running \texttt{make generate}, the material between the opening and
731 the closing line is replaced by the output of the command.
732
733 For instance, in the following snippet the material between the comments
734 was inserted by \texttt{make generate}:
735
736 \begin{lstlisting}[language=TeX]
737 % GENERATED: echo This is generated content
738 This is generated content
739
740 % /GENERATED
741 \end{lstlisting}
742
743 \texttt{make generate} will not replace any content in the document source
744 if even just one command fails to execute as indicated by the
745 command's return code; it is transactional in this sense.
746
747 We mention in passing that in addition to allowing arbitrary shell
748 commands, \texttt{update\_generated.py} has a facility that allows
749 calling special python functions from within documents: Commands starting
750 with an exclamation mark (``!'') are interpreted as such calls to
751 functions defined within \texttt{update\_generated.py}.
752 Currently, the only such builtin command extracts documentation from a
753 live \texttt{TAP\_SCHEMA}\footnote{It is in use in the
754 \href{https://volute.g-vo.org/svn/trunk/projects/registry/discovercollections}{Discovering
755 Data Collections Within Services} IVOA Note.}. For now, this should be
756 considered an unstable experiment, but authors wanting to try inline
757 python for content generation are welcome to do so.
758
759 \ivoatex will never execute \texttt{update\_generated.py} as part of a
760 dependency chain; it is intended that \texttt{make generate} must always
761 been manually triggered. On the one hand, this is because its
762 dependencies cannot be generically modelled, given that arbitrary
763 commands can be executed. Document authors are also discouraged from
764 providing such dependency information -- it is fairly common that
765 content generation depends on the availability of external resources
766 (e.g., databases or network services), and a document build should not
767 fail just because these are unavailable.
768
769 We mention in passing that generated content puts potentially executable
770 material into documents, which is of course an attack vector for
771 malicious software. However, calling \texttt{make update} is no
772 additional security risk, as presumably the author of the makefile is
773 identical to the document author, and the makefile can already contain
774 arbitrary commands that would be executed on the calling user's behalf.
775
776 \subsubsection{XML Schema documentation}
777
778 A common use case for this facility is building documentation from XML
779 schema files. To support it, \ivoatex\ comes with an XSLT file
780 \texttt{schemadoc.xslt} that generates \TeX~source for one type, the
781 name of which is passed in through a stylesheet parameter named
782 \texttt{destType}. Since \ivoatex\ itself depends on xsltproc, the
783 recommended way to use the facility looks like this:
784
785 \lstinputlisting[language=tex,basicstyle=\footnotesize]{schemainclusion.tex}
786
787 Note that currently the entire command line must be in one physical line
788 in the file, so the recommendation to keep source lines shorter than 72
789 characters clearly does not apply here.
790
791
792 \section{Customisation and Development}
793 \label{sect:impl}
794
795 This section discusses aspects of \ivoatex\ that are more technical in
796 nature. Authors with a modicum of \TeX\ expertise are nevertheless
797 encouraged to read it.
798
799 \subsection{Technical Overview}
800
801 The central files in \ivoatex\ processing are
802
803 \begin{bigdescription}
804 \item[ivoa.cls] The class file, inheriting from \LaTeX's article class.
805 The file defines the markup rules for PDF processing, including
806 titlepage generation and extra macros and environments. Its content is
807 ignored for HTML generation.
808
809 \item[tthdefs.tex] This file protects its contents from normal \TeX\
810 processing by a \verb|\iftth| conditional. This way, only tth sees
811 definitions made here. Each special feature defined in \texttt{ivoa.cls}
812 has a counterpart here, giving rules for translation to HTML. This
813 usually encompasses emitting some HTML before and after the argument of
814 a TeX construct, where material between \verb|\begin{html}| and
815 \verb|\end{html}| is included literally in the HTML document.
816
817 \item[tth-ivoa.xslt] An XSLT stylesheet that postprocesses tth's output
818 and performs some operations that would be inconvenient to implement in
819 \texttt{tthdefs.tex}, in particular for the formatting of the opening
820 material.
821
822 \item[Makefile] This makefile is included by the user makefile in the
823 document directory proper. It defines the rules given above as well as
824 some extra housekeeping rules like package building and building tth
825 from its source.
826
827 \end{bigdescription}
828
829 \subsection{Semantic Markup}
830
831 In order to make it support rich, semantic markup, \ivoatex\ needs to be
832 continuously developed. In particular, it is good practice to define
833 macros for marking up values of certain datatypes, as with \ivoatex's
834 \texword{xmlel} and \texword{vorent}.
835 Thus, whenever a document has multiple
836 instances of such values, authors should define macros and use these.
837 For instance, RegTAP deals with lots of concepts from its own
838 database schema and hence has
839 \begin{lstlisting}[language=TeX]
840 \definecolor{rtcolor}{rgb}{0.15,0.4,0.3}
841 \newcommand{\rtent}[1]{\texttt{\color{rtcolor} #1}}
842 \end{lstlisting}
843 in its document preamble to
844 define markup for ``RegTAP entity'', whereas
845 this note, as it mentions many words with a special meaning to \TeX, has
846 \begin{lstlisting}[language=TeX]
847 \definecolor{texcolor}{rgb}{0.4,0.1,0.1}
848 \newcommand{\texword}[1]{\texttt{\color{texcolor} #1}}
849 \end{lstlisting}
850 Such macros will be included in \ivoatex\ itself rather than an
851 individual document's preamble as they prove useful in multiple
852 documents.
853
854 \subsection{Custom Macros and Environments}
855
856 The tth translator used by \ivoatex\ ignores \texword{usepackage}. Many
857 common packages are natively supported, but those that are not in
858 general need specific handling, and sometimes support is somewhat spotty.
859 For instance, the \texword{nolinkurl}
860 macro is not supported natively by tth, and hence in
861 \texttt{tthdefs.tex} there is
862 \begin{lstlisting}[language=TeX]
863 \newcommand{\nolinkurl}[1]{%
864 \special{html:<span class='nolinkurl'>}#1\special{html:</span>}}
865 \end{lstlisting}
866
867 When a document requires special markup, it is likely that
868 different implementations will be necessary for PDF and HTML output.
869 Using \texword{iftth} the implementations for the current output mode
870 can be selected (without the \texword{newif} mentioned in the tth
871 documentation, as that is already performed in \texttt{tthdefs.tex}.
872
873 For instance, RegTAP has many inline tables that need special spacing
874 the PDF rendering, whereas normal tables will do for them
875 in HTML. It therefore
876 has in its preamble the definitions
877 \begin{lstlisting}
878 \iftth
879 \newenvironment{inlinetable}{}{}
880 \else
881 \newenvironment{inlinetable}{\vskip 1ex\vfil
882 \penalty8000\vfilneg%
883 \hbox to\hsize\bgroup\hss}
884 {\hss\egroup\vspace{8pt}}
885 \fi
886 \end{lstlisting}
887
888
889 \subsection{Custom CSS}
890
891 If you find you need custom CSS to fix HTML formatting, you should
892 probably talk to \ivoatex's authors first. There are, however,
893 legitimate cases when something needs extra styling in HTML that simply
894 somes out right in the PDF output. In such cases, a standard CSS file can
895 be added to a repository (it must then also be added to \texttt{SOURCES}
896 in the Makefile in order for it to be delivered with the document
897 package).
898
899 The document itself would then use the \ivoatex's \texword{customcss}
900 macro in its preamble with the CSS file name as an argument. For
901 example, the source for this document says
902
903 \begin{lstlisting}
904 \iftth
905 \newcommand{\comicstuff}[1]{
906 \begin{html}<span class="comic">#1</span>\end{html}}
907 \else
908 \newcommand{\comicstuff}[1]{(HTML exclusive material)}
909 \fi
910 \end{lstlisting}
911
912 \noindent in its preamble. With this (and a CSS file that can be inspected in the
913 distribution),
914
915 \begin{lstlisting}
916 \comicstuff{If this is comic sans, your webbrowser is permissive.}
917 \end{lstlisting}
918
919 \noindent becomes:
920 \comicstuff{If this is comic sans, your webbrowser is permissive.}
921
922 \subsection{Migration from Ivoadoc}
923
924 To ease migration from documents authored in ivoadoc, \ivoatex\ comes
925 with an XSLT stylesheet writing out a starting point for an ivoatex
926 version. While many desirable features (e.g., extraction of titlepage
927 information) are not implemented and translated tables are incomplete,
928 the styleheet should still save time. For XHTML-compliant ivoadoc
929 sources, the stylesheet is used like this:
930
931 \begin{lstlisting}[language=sh]
932 xsltproc ivoatex/fromivoadoc.xslt ivoadoc_source.html
933 \end{lstlisting}
934
935
936 \section{Desirable Features to be Implemented}
937
938 Frequently, certain parts of technical documents should be
939 machine-ge\-ne\-ra\-ted. Examples of such parts include the automatic
940 extraction of XML schema snippets and their inclusion in the document
941 as provided by ivoadoc, or the TAP\_SCHEMA-derived tables of column
942 metadata in RegTAP. As RegTAP shows, machine-generated content can
943 already be used within \ivoatex, but it would certainly be preferable if
944 some default framework for doing this were provided.
945
946 The inclusion of IVOA architecture diagrams as bitmaps prepared by the
947 TCG chair is unsatisfactory on both technical and organisational
948 grounds. A common system allowing their easy generation based on SVG or
949 one of the \TeX-based graphics engines should be written.
950
951 A major drawback of \ivoatex's HTML output is that paragraphs are not actually
952 marked up as such. Due to the \TeX\ processing model, their
953 reconstruction is non-trivial. Hence in the generated HTML,
954 source-level paragraphs are rendered as text nodes separated by empty
955 HTML paragraph elements. It would probably be possible to rectify this
956 in the XSLT postprocessing.
957
958
959 \appendix
960 \section{Changes from Previous Versions}
961
962 \subsection{Changes from Version 1.0}
963 \begin{enumerate}
964 \item Changed google code URLs to volute.g-vo.org ones.
965 \item Documented new facilities for generating material, with extra
966 focus on auto-documenting XML schema.
967 \end{enumerate}
968 % these would be subsections "Changes from v. WD-..."
969 % Use itemize environments.
970
971
972 \bibliography{ivoatex/ivoabib}
973
974
975 \end{document}

Properties

Name Value
svn:keywords Date Rev URL

msdemlei@ari.uni-heidelberg.de
ViewVC Help
Powered by ViewVC 1.1.26