/[volute]/trunk/projects/registry/regtap/RegTAP.tex
ViewVC logotype

Contents of /trunk/projects/registry/regtap/RegTAP.tex

Parent Directory Parent Directory | Revision Log Revision Log


Revision 3089 - (show annotations)
Wed Oct 7 08:24:56 2015 UTC (6 years ago) by msdemlei
File MIME type: application/x-tex
File size: 120959 byte(s)
Properly declared extra dependencies in the Makefile.

(also, a minute fix in an example.  I'm leaving this in although in this
way the arXiv submission will have a tiny difference from what's in the ivoa
doc repo.  Sue me)


1 \documentclass[11pt,a4paper]{ivoa}
2 \input tthdefs
3
4 \tolerance=6000
5 \hbadness=6000
6
7 \usepackage[utf8]{inputenc}
8 \usepackage{longtable}
9 \usepackage{listings}
10 \lstloadlanguages{XML,SQL}
11
12 \definecolor{rtcolor}{rgb}{0.15,0.4,0.3}
13 \definecolor{tapcolor}{rgb}{0.4,0.1,0.1}
14
15 \newcommand{\rtent}[1]{\texttt{\color{rtcolor} #1}}
16 \newcommand{\tapent}[1]{\texttt{\color{tapcolor} #1}}
17
18
19 \ivoagroup{Registry}
20
21 \author[http://www.ivoa.net/cgi-bin/twiki/bin/view/IVOA/MarkusDemleitner]{Markus Demleitner}
22 \author[http://www.ivoa.net/cgi-bin/twiki/bin/view/IVOA/PaulHarrison]{Paul Harrison}
23 \author[http://www.ivoa.net/cgi-bin/twiki/bin/view/IVOA/MarcoMolinaro]{Marco Molinaro}
24 \author[http://www.ivoa.net/cgi-bin/twiki/bin/view/IVOA/GretchenGreene]{Gretchen Greene}
25 \author[http://www.ivoa.net/cgi-bin/twiki/bin/view/IVOA/TheresaDower]{Theresa Dower}
26 \author[http://wiki.ivoa.net/twiki/bin/view/IVOA/MenelaosPerdikeas]{Menelaos Perdikeas}
27
28
29 \editor{Markus Demleitner}
30
31 \previousversion[http://www.ivoa.net/documents/RegTAP/20141030]{http://www.ivoa.net/documents/RegTAP/20141030}
32 \previousversion[http://www.ivoa.net/documents/RegTAP/20140227]{http://www.ivoa.net/documents/RegTAP/20140227}
33 \previousversion[http://www.ivoa.net/documents/RegTAP/20140627]{http://www.ivoa.net/documents/RegTAP/20140627}
34 \previousversion[http://www.ivoa.net/documents/RegTAP/20140227]{http://www.ivoa.net/documents/RegTAP/20140227}
35 \previousversion[http://www.ivoa.net/documents/RegTAP/20131203]{http://www.ivoa.net/documents/RegTAP/20131203}
36 \previousversion[http://www.ivoa.net/documents/RegTAP/20130909]{http://www.ivoa.net/documents/RegTAP/20130909}
37 \previousversion[http://www.ivoa.net/documents/RegTAP/20130411]{http://www.ivoa.net/documents/RegTAP/20130411}
38 \previousversion{Internal Working Draft 2013-03-05 (Volute
39 rev. 2011)}
40 \previousversion{ Internal Working Draft 2012-11-12 (Volute
41 rev. 1864)}
42
43
44 \title{IVOA Registry Relational Schema}
45
46 \begin{document}
47
48 \begin{abstract}
49 Registries provide a mechanism with which VO applications can
50 discover and select resources -- first and foremost data and
51 services -- that are relevant for a particular scientific problem.
52 This specification defines an interface for searching this resource
53 metadata based on the IVOA's TAP protocol. It specifies a set of tables
54 that comprise a useful subset of the information contained in the
55 registry records, as well as the table's data content in terms of the
56 XML VOResource data model. The general design of the system is geared
57 towards allowing easy authoring of queries.
58 \end{abstract}
59
60
61 % class=``head''
62
63 % HTML section start
64
65 \section{Introduction}
66
67 \label{intro}
68
69 In the Virtual Observatory (VO), registries provide a means for
70 discovering useful resources, i.e., data and services. Individual
71 publishers offer the descriptions for their resources (``resource
72 records'') in publishing registries. At the time of writing, there are
73 roughly 14000 such resource records active within the VO, originating
74 from about 20 publishing registries.
75
76 The protocol spoken by these
77 publishing registries, OAI-PMH, only allows restricting queries by
78 modification date and identifier and is hence not suitable for data discovery.
79 Even if it were, data discovery would at least be fairly time consuming if
80 each client had to query dozens or, potentially, hundreds of
81 publishing registries.
82
83 To enable efficient data discovery nevertheless, there are services harvesting the
84 resource records from the publishing registries and offering rich query
85 languages.
86 The IVOA Registry
87 Interfaces specification \citep{std:RI1} defined, among other aspects of
88 the VO registry system, such an interface
89 using SOAP and an early draft of an XML-based query language.
90
91 This document provides an update to the query (``searchable registry'') part
92 of that specifiation (chapter 2), aimed towards
93 usage with current VO standards, in particular TAP \citep{std:TAP}
94 and ADQL \citep{std:ADQL}. It follows the model of ObsCore
95 \citep{std:OBSCORE} of defining a representation of a data model
96 within a relational database. In this case, the data model is a
97 simplification of the VO's resource metadata interchange representation,
98 the VOResource XML format \citep{std:VOR}. The simplification
99 yields 13 tables. For each table, TAP\_SCHEMA metadata is given together
100 with rules for how to fill these tables from VOResource-serialized
101 metadata records as well as conditions on foreign keys and
102 recommendations on indexes.
103
104 The resulting set of tables has a modest size by today's standards,
105 but is still non-trivial. The largest table, \rtent{table\_column},
106 has about half a million rows at the time of writing.
107
108 The architecture laid out here allows client applications to perform ``canned''
109 queries on behalf of their users as well as complex queries formulated
110 directly by advanced users, using the same TAP clients they employ to
111 query astronomical data servers.
112
113
114 % HTML section start
115
116 \subsection{Terminology}
117
118 \label{terms}
119
120 The set of tables and their metadata specified here, together with
121 the mapping from VOResource (``ingestion rules'') is collectively
122 called ``relational registry schema'' or ``relational registry'' for
123 short.
124
125 The specificiation additionally talks about how to embed these into TAP
126 services, gives additional user defined functions, talks about
127 discovering compliant services, etc. Since all this is tightly coupled
128 to the ``relational registry'' as defined above, we do not
129 introduce a new term for it. Hence, the entire standard is now known as
130 ``IVOA registry relational schema''.
131
132 Historically, we intended to follow the ObsCore/ObsTAP model and
133 talked about RegTAP. As changing this acronym is technically painful
134 (e.g., identifiers and URLs would need to be adapted), we kept it even after
135 the distinction between the schema and its mapping on the one hand and
136 its combination with a TAP service on the other went away. This
137 means that the official acronym for ``IVOA registry relational schema'' is
138 RegTAP. This aesthetic defect seems preferable to causing actual
139 incompatibilities.
140
141
142 % HTML section ends
143
144 % subsection terms
145 % HTML section start
146
147 \subsection{The Relational Registry within the VO Architecture}
148
149 \label{rolewithinivoa}
150
151
152 \begin{figure}[thm]
153 \begin{center}
154 \includegraphics[width=0.9\textwidth]{RegTAP-arch.png}
155 \end{center}
156
157 \caption{IVOA Architecture
158 diagram with the IVOA Registry Relational Specification (tagged with
159 ``Relational Registry'') and the related standards marked up.}
160 \end{figure}
161
162 This specification directly relates to other VO standards in the
163 following ways:
164
165
166 \begin{description}
167 \item[VOResource, v1.03 \citep{std:VOR}] VOResource sets the foundation for a formal definition of the data
168 model for resource records via its schema definition. This document
169 refers to concepts laid down there via xpaths \citep{std:XPATH}.
170 \item[VODataService, v1.1 \citep{std:VODS11}] The VODataService standard
171 describes several concepts and resource types extending
172 VOResource's data model, including
173 tablesets, data services and data
174 collections. These concepts and types are reflected in the database
175 schema. Again xpaths link this specification and VODataService.
176 \item[Other Registry Extensions]Registry extensions are VO standards
177 defining how particular resources (e.g., Standards) or capabilities
178 (e.g., IVOA defined interfaces) are described. Most aspects
179 introduced by them are reflected in the \rtent{res\_detail} table using
180 xpaths into the registry documents.
181 This document should not in general need updates
182 for registry extension updates. For completeness, we note the
183 versions current as of this specification: SimpleDALRegExt 1.0
184 \citep{std:DALREGEXT},
185 StandardsRegExt 1.0 \citep{std:STDREGEXT}, TAPRegExt 1.0
186 \citep{std:TAPREGEXT}, Registry Interfaces 1.0 \citep{std:RI1}.
187 \item[TAP, v1.0 \citep{std:TAP}]The queries against the schema defined in the present document, and the results of
188 these queries, will usually be transported using the Table Access
189 Protocol TAP. It also allows discovering
190 local additions to the registry relations via TAP's metadata publishing
191 mechanisms.
192 \item[IVOA Identifiers, v1.12 \citep{std:VOID}]IVOA identifiers are
193 essentially the primary keys within the VO
194 registry; as such, they are actual primary keys of the central table of
195 the relational registry. Also, the notion of an authority as laid down
196 in IVOA Identifiers plays an important role as publishing registries can
197 be viewed as a realization of a set of authorities.
198
199 \end{description}
200
201 This standard also relates to other IVOA standards:
202
203
204 \begin{description}
205 \item[ADQL, v2.0 \citep{std:ADQL}]The rules for ingestion are designed to allow
206 easy queries given the constraints of ADQL 2.0. Also,
207 we give four functions that extend ADQL using the
208 language's built-in facility for user-defined functions.
209 \end{description}
210
211
212 % HTML section ends
213
214 % subsection rolewithinivoa
215 % HTML section ends
216
217 % section introduction
218 % HTML section start
219
220 \section{Design Considerations}
221
222 \label{design}
223
224 In the design of the tables, the goal has been to preserve as much of
225 VOResource and its extensions, including the element names, as
226 possible.
227
228 An overriding consideration has been, however, to make natural joins
229 between the tables behave usefully, i.e., to actually combine rows
230 relevant to the same entity (resource, table, capability, etc.).
231 To disambiguate column names that name the same concept on different
232 entities (name, description, etc.) and would therefore interfere with
233 the natural join, a shortened tag for the source object
234 is prepended to the name. Thus, a \vorent{description} element within
235 a resource ends up in a column named
236 \rtent{res\_description}, whereas the same element from a
237 \vorent{capability} becomes \rtent{cap\_description}.
238
239 We further renamed some columns and most tables
240 with respect to their VOResource
241 counterparts to avoid clashes with reserved words in popular database
242 management systems. The alternatives would have been to either recommend
243 quoting them or burden ADQL translation layers with the task of
244 automatically converting them to delimited identifiers. Both
245 alternatives seemed more confusing and less robust than the renaming
246 proposed here.
247
248 Furthermore, camel-case identifiers have been converted to
249 underscore-separated ones (thus, \vorent{standardId} becomes
250 \rtent{standard\_id}) to have all-lowercase column names; this saves
251 potential headache if users choose to reference the columns using SQL
252 delimited identifiers. Dashes in VOResource attribute names are
253 converted to underscores, too, with the exception of
254 \vorent{ivo-id}, which is just rendered \rtent{ivoid}.
255
256 Another design goal of this specification has been that different registries
257 operating on the same set of registry records will return identical responses
258 for most queries; hence, we try to avoid relying on features left not
259 defined by ADQL (e.g., the case sensitivity of string matches). However,
260 with a view to non-uniform support for information retrieval-type
261 queries in database systems, the \rtent{ivo\_hasword} user defined
262 function is not fully specified here; queries employing it may yield
263 different results on different implementations, even if they operate on
264 the same set of resource records.
265
266
267 % HTML section ends
268
269 % subsection design
270 % HTML section start
271
272 \section{Primary Keys}
273
274 \label{primarykeys}
275
276 The primary key in the Registry as an abstract concept is a resource
277 record's IVORN. Hence, for all tables having primary keys at all, the
278 \rtent{ivoid} column is part of its primary key. This
279 specification does not require implementations to actually declare
280 primary keys in the underlying database, and no aspect of user-visible
281 behavior depends on such explicit declarations; in particular, this
282 specification makes no requirements on the contents of
283 \tapent{tap\_schema.keys}.
284
285 We nevertheless make recommendations on explicit primary keys, as
286 we expect definitions according to our recommendations will enhance
287 robustness of services.
288
289 In several RegTAP tables -- \rtent{capability},
290 \rtent{res\_schema}, \rtent{res\_table}, and
291 \rtent{interface} -- artificial primary keys are necessary, as
292 in VOResource XML sibling elements are not otherwise distinguished. To
293 allow such artificial primary keys, a column is added to each table, the
294 name of which ends in \texttt{\_index} (\rtent{cap\_index},
295 \rtent{schema\_index}, \rtent{table\_index}, and
296 \rtent{intf\_index}).
297
298 The type and content of these \texttt{X\_index} columns is
299 implementation-defined, and clients must not make assumptions on their
300 content except that the pair \rtent{ivoid}, \texttt{X\_index} is a primary
301 key for the relation (plus, of course, that references from other tables
302 correctly resolve). In the tables of columns given below, the
303 \texttt{X\_index} columns have ``(key)'' given for type. Implementors
304 obviously have to insert whatever ADQL type is appropriate for their
305 choice or \texttt{X\_index} implementation.
306
307 Obvious implementations for \texttt{X\_index} include having
308 \texttt{X\_index} enumerate the sibling elements or using some sort
309 of UUID.
310
311
312 % HTML section ends
313
314 % HTML section start
315
316 \section{Notes on string handling}
317
318 \label{stringnorm}
319
320 In the interest of consistent behavior between different RegTAP
321 implementations regardless of their technology choices, this section
322 establishes some rules on the treatment of strings -- both those
323 obtained from attributes and those obtained from element
324 content -- during ingestion from VOResource XML to database
325 tables.
326
327
328 % HTML section start
329
330 \subsection{Whitespace normalization}
331
332 \label{whitenorm}
333
334 Most string-valued items in VOResource and extensions are of type
335 \texttt{xs:to\-ken}, with the clear intent that whitespace in them is
336 to be normalized in the sense of XML schema. For the few exceptions
337 that actually are directly derived from xs:string (e.g.,
338 \vorent{vstd:EndorsedVersion}, \vorent{vs:Waveband}) it does not
339 appear that the intent regarding whitespace is different.
340
341 In order to provide reliable querying and simple rules for ingestors
342 even when these do not employ schema-aware XML parsers, this standard
343 requires that during ingestion, leading and trailing whitespace MUST be
344 removed from all strings; in particular, there are no strings consisting
345 exclusively of whitespace in RegTAP. The treatment of internal
346 whitespace is implementation-defined. This reflects the expectation
347 that, whereever multi-word items are queried, whitespace-ignoring
348 constraints will be used (e.g., LIKE-based regular expressions or the
349 \rtent{ivo\_hasword} user defined function defined below).
350
351
352 % HTML section ends
353
354 % subsection whitenorm
355 % HTML section start
356
357 \subsection{NULL/Empty string normalization}
358
359 \label{nullnorm}
360
361 While empty strings and NULL values are not usually well
362 distinguished in VO practice -- as reflected in the conventional
363 TABLEDATA and BINARY serializations of VOTable -- , the distinction
364 must be strictly maintained in the database tables to ensure
365 reproduceable queries across different RegTAP implementations.
366
367 Ingestors therefore MUST turn empty strings (which, by section \ref{whitenorm}, include strings consisting of whitespace
368 only in VOResource's XML serialization) into NULL values in the
369 database. Clients expressing constraints on the presence (or absence)
370 of some information must therefore do so using SQL's \texttt{IS NOT NULL}
371 (or \texttt{IS NULL}) operators.
372
373
374 % HTML section ends
375
376 % subsection nullnorm
377 % HTML section start
378
379 \subsection{Case normalization}
380
381 \label{casenorm}
382
383 ADQL has no operators for case-insensitive matching of strings. To
384 allow for robust and straightforward queries nevertheless, most columns
385 containing values not usually intended for display are required to be
386 converted to lower case; in the table descriptions below, there are
387 explicit requirements on case normalization near the end of each
388 section. This is particularly important when the entities to be
389 compared are defined to be case-insensitive (e.g., UCDs, IVORNs).
390 Client software that can inspect user-provided arguments (e.g., when
391 filling template queries) should also convert the respective fields to
392 lower case.
393
394 This conversion MUST cover all ASCII letters, i.e., A through Z.
395 The conversion SHOULD take place according to
396 algorithm R2 in section 3.13, ``Default Case Algorithms'' of the Unicode
397 Standard
398 \citep{std:UNICODE}. In practice, non-ASCII characters are not expected
399 to occur in columns for which lowercasing is required.
400
401 Analogously, case-insensitive comparisons as required by some of the
402 user-defined functions for the relational registry MUST compare
403 the ASCII letters without regard for case. They SHOULD compare according
404 to D144 in the Unicode Standard.
405
406
407 % HTML section ends
408
409 % subsection casenorm
410 % HTML section start
411
412 \subsection{Non-ASCII characters}
413
414 \label{utfreq}
415
416 Neither TAP nor ADQL mention non-ASCII in service parameters -- in
417 particular the queries -- or returned values. For RegTAP, that is
418 unfortunate, as several columns will contain relevant non-ASCII
419 characters. Columns for which extra care is necessary include all
420 descriptions, \rtent{res\_title} and \rtent{creator\_seq} in
421 \rtent{rr.resource}, as well as \rtent{role\_name} and
422 \rtent{street\_address} in \rtent{rr.res\_role}.
423
424 RegTAP implementations SHOULD be able to faithfully represent all
425 characters defined in the latest version of the Unicode standard
426 \citep{std:UNICODE} at
427 any given time and allow querying using them (having support for UTF-8
428 in the database should cover this requirement) for at least the fields
429 mentioned above.
430
431 On VOResource ingestion, non-ASCII characters that a service cannot
432 faithfully store MUST be replaced by a question mark character (``?'').
433
434 RegTAP services MUST interpret incoming ADQL as encoded in UTF-8,
435 again replacing unsupported characters with question marks.
436
437 We leave character replacement on result generation unspecified, as
438 best-effort representations (e.g., ``Angstrom'' instead of ``Ångström'')
439 should not impact interoperability but significantly improve user
440 experience over consistent downgrading. In VOTable output,
441 implementations SHOULD support full Unicode in at least the fields
442 enumerated above. Clients are advised to retrieve results in VOTable or
443 other encoding-aware formats.
444
445 Note that with VOTable 1.3, non-ASCII in char-typed fields, while
446 supported by most clients in TABLEDATA serialization, is technically
447 illegal; it is essentially undefined in other serializations. To
448 produce standards-compliant VOTables, columns containing non-ASCII must
449 be of type unicodeChar. We expect that future versions of VOTable will
450 change the definitions of char and unicodeChar to better match modern
451 standards and requirements. RegTAP implementors are encouraged to take
452 these up.
453
454
455 % HTML section ends
456
457 % subsection utfreq
458 % HTML section ends
459
460 % section stringnorm
461 % HTML section start
462
463 \section{QNames in VOResource attributes}
464
465 \label{qnameatts}
466
467 VOResource and its extensions make use of XML QNames in attribute
468 values, most prominently in \texttt{xsi:type}. The standard
469 representation of these QNames in XML instance documents makes use of an
470 abbreviated notation employing prefixes declared using the xmlns mechanism
471 as discussed in \citet{std:XMLNS}. Within an ADQL-exposed database, no
472 standard mechanism exists that could provide a similar mapping of URLs
473 and abbreviations. The correct way to handle this problem would thus be
474 to have full QNames in the database (e.g.,
475 \texttt{{http://www.ivoa.net/xml/ConeSearch/v1.0}ConeSearch} for the
476 canonical \vorent{cs:ConeSearch}). This, of course, would make for
477 excessively tedious and error-prone querying.
478
479 For various reasons, VOResource authors have always been encouraged
480 to use a set of ``standard'' prefixes. This allows an easy and, to users,
481 unsurprising exit from the problem of the missing xmlns declarations:
482 For the representation of QNames within the database, these recommended
483 prefixes are now mandatory. Future VOResource extensions define their
484 mandatory prefixes themselves.
485
486 Following the existing practice, minor version changes are not in
487 general reflected in the recommended prefixes -- e.g., both VODataService
488 1.0 and VODataService 1.1 use \texttt{vs:}. For reference, here is
489 a table of XML namespaces and prefixes for namespaces relevant to this
490 specification:
491
492
493 \begin{inlinetable}
494 \begin{tabular}{ll}
495 cs&http://www.ivoa.net/xml/ConeSearch/v1.0\\
496 dc&http://purl.org/dc/elements/1.1/\\
497 oai&http://www.openarchives.org/OAI/2.0/\\
498 ri&http://www.ivoa.net/xml/RegistryInterface/v1.0\\
499 sia&http://www.ivoa.net/xml/SIA/v1.0\\
500 sia&http://www.ivoa.net/xml/SIA/v1.1\\
501 slap&http://www.ivoa.net/xml/SLAP/v1.0\\
502 ssap&http://www.ivoa.net/xml/SSA/v1.0\\
503 ssap&http://www.ivoa.net/xml/SSA/v1.1\\
504 tr&http://www.ivoa.net/xml/TAPRegExt/v1.0\\
505 vg&http://www.ivoa.net/xml/VORegistry/v1.0\\
506 vr&http://www.ivoa.net/xml/VOResource/v1.0\\
507 vs&http://www.ivoa.net/xml/VODataService/v1.0\\
508 vs&http://www.ivoa.net/xml/VODataService/v1.1\\
509 vstd&http://www.ivoa.net/xml/StandardsRegExt/v1.0\\
510 xsi&http://www.w3.org/2001/XMLSchema-instance\\
511
512 \end{tabular}
513 \end{inlinetable}
514
515 % HTML section ends
516
517 % HTML section start
518
519 \section{Xpaths}
520
521 \label{vorutypes}
522
523 This specification piggybacks on top of the well-established
524 VOResource standard. This means that it does not define a full data model,
525 but rather something like a reasonably query-friendly view of a partial
526 representation of one. The link between the actual data model, i.e.,
527 VOResource and its extensions as defined by the XML Schema documents, and
528 the fields within this database schema, is provided by
529 xpaths, which are here slightly abbreviated for both brevity and
530 generality.
531
532 All xpaths given in this specifiation are assumed to be relative to
533 the enclosing \vorent{vr:Resource} element; these are called
534 ``resource xpaths'' in the following. If resource xpaths are to be
535 applied to an OAI-PMH response, the Xpath expression
536 \texttt{*/*/*/oai:metadata/ri:Resource} must be prepended to it,
537 with the canonical prefixes from section \ref{qnameatts} implied. The resource xpaths themselves
538 largely do not need explicit namespaces since VOResource elements are by
539 default unqualified. Elements and attributes from non-VOResource
540 schemata have the canonical namespace prefixes, which in this
541 specification only applies to several \texttt{xsi:type} attribute
542 names.
543
544 Some tables draw data from several different VOResource elements.
545 For those, we have introduced an extended syntax with additional
546 metacharacters \verb$($, \verb$)$, and \verb$|$,
547 where the vertical bar denotes an
548 alternative and the parentheses grouping. For instance, our notation
549 \texttt{/(tableset/schema/|)table/} corresponds to the two xpaths
550 \texttt{/table} and \texttt{/tableset/schema/table}.
551
552 Within the Virtual Observatory, the link between data models and
553 concrete data representations is usually made using utypes.
554 Since VOResource is directly modelled
555 in XML Schema, the choice of XPath as the bridging formalism is
556 compelling, though, and utypes themselves are not necessary for the
557 operation of a TAP service containing the relational registry.
558 TAP, however, offers fields for utypes in its TAP\_SCHEMA. Since they
559 are not otherwise required, this specification takes the liberty of
560 using them to denote the xpaths.
561
562 In the metadata for tables and columns below, the utypes given are
563 obtained from the xpaths by simply prepending them with
564 \texttt{xpath:}. To avoid repetition, we allow relative xpaths:
565 when the xpath in a column utype does not start with a slash, it is
566 understood that it must be concatenated with the table utype to obtain
567 the full xpath.
568
569 For illustration, if a table has a utype of
570 $$\texttt{xpath:/capability/interface/}$$ and a column within this table
571 has a utype of $$\texttt{xpath:accessURL/@use},$$ the resulting resource
572 xpath would come out to be
573 $$\texttt{/capability/interface/accessURL/@use};$$ to match this in an
574 OAI-PMH response, the XPath would be
575 $$\texttt{\small
576 */*/*/oai:metadata/ri:Resource/capability/interface/accessURL/@use}.$$
577
578
579 While clients MUST NOT rely on these utypes in either
580 \tapent{TAP\_SCHEMA} or the
581 metadata delivered with TAP replies, service operators SHOULD provide them, in
582 particular when there are local extensions to the relational registry in their
583 services. Giving xpaths for extra columns and tables helps human
584 interpretation of them at least when the defining schema files are
585 available.
586
587 Resource xpaths are also used in the \rtent{res\_details} table (section
588 \ref{table_res_detail}). These are normal xpaths
589 (although again understood relative to the enclosing Resource element),
590 which, in particular, means that they are case sensitive. On the other
591 hand, to clients they are simply opaque strings, i.e., clients cannot
592 just search for any xpaths into VOResource within \rtent{res\_details}.
593
594
595 % HTML section ends
596
597 % subsection vorutypes
598 % HTML section start
599
600 \section{Discovering Relational Registries}
601
602 \label{registration}
603
604 The relational registry can be part of any TAP service. The presence
605 of the tables discussed here is indicated by declaring support for the
606 data model \texttt{Registry 1.0} with the IVORN
607 $$\texttt{ivo://ivoa.net/std/RegTAP\#1.0}$$ in the service's
608 capabilities (cf.~\citet{std:TAPREGEXT}). Technically, this
609 entails adding
610
611
612 \begin{verbatim}
613 <dataModel ivo-id="ivo://ivoa.net/std/RegTAP#1.0"
614 >Registry 1.0</dataModel>
615 \end{verbatim}
616
617 as a child of the capability element with the type
618 \vorent{tr:TableAccess}.
619
620 A client that knows the access URL of one TAP service containing
621 a relational
622 registry can thus discover all other services exposing one. The \href{#example-findregtap}{``Find all TAP endpoints offering the
623 relational registry''} example in section
624 \ref{sample_queries} shows a query that does
625 this.
626
627 It is recommended to additionally register a relational registry as a
628 VODataService data collection and connect this and the TAP services
629 with a pair of service-for/served-by relations. This allows giving more
630 metadata on the data content like, for example, the frequency of
631 harvesting.
632
633 Services implementing this data model that do not (strive to) offer
634 the full data content of the VO registry (like domain-specific
635 registries or experimental systems) MUST NOT declare the above data
636 model in order to not invite clients expecting the VO registry to send
637 queries to it.
638
639
640 % HTML section ends
641
642 % subsection registration
643 % HTML section start
644
645 \section{VOResource Tables}
646
647 \label{vortables}
648
649 In the following table descriptions, the names of tables
650 (cf.~Table \ref{table:dm}) and columns
651 are normative and MUST be used as given, and all-lowercase. On the
652 values in the \tapent{utype} columns within \tapent{TAP\_SCHEMA},
653 see section \ref{vorutypes}. All columns defined in
654 this document MUST have a 1 in the \tapent{std} column of the
655 \tapent{TAP\_SCHEMA.table\_columns} table. Unless otherwise
656 specified, all values of ucd and unit in
657 \tapent{TAP\_SCHEMA.table\_columns} are NULL for columns defined here.
658 Descriptions are not normative (as given, they usually are taken from
659 the schema files of VOResource and its extensions with slight
660 redaction). Registry operators MAY provide additional columns in their
661 tables, but they MUST provide all columns given in this
662 specification.
663
664 All table descriptions start out with brief remarks on the
665 relationship of the table to the VOResource XML data model. Then, the
666 columns are described in a selection of TAP\_SCHEMA metadata. For each
667 table, recommendations on explicit primary and foreign keys as well as
668 indexed columns are given, where it is understood that primary and
669 foreign keys are already indexed in order to allow efficient joins;
670 these parts are not normative, but operators should ensure decent
671 performance for queries assuming the presence of the given indexes and
672 relationships. Finally, lowercasing requirements
673 (normative) are given.
674
675
676 \begin{figure}
677
678 \includegraphics[width=\textwidth]{schema.pdf}
679 \caption{A sketch of the
680 Relational Registry schema, adapted from \citet{regtap-adass}.
681 Only the columns considered
682 most interesting for client use are shown. Arrows indicate foreign
683 key-like relationships.}
684 \end{figure}
685
686
687 % GENERATED: gettables.sh
688
689 \begin{table}[t]
690 \small
691 \hbox to\hsize{\hss
692 \begin{tabular}{p{0.35\textwidth}p{0.64\textwidth}}
693 \sptablerule
694 \textbf{Name and UType}&\textbf{Description}\\
695 \sptablerule
696 rr.capability\hfil\break
697 \makebox[0pt][l]{\scriptsize\ttfamily xpath:/capability/}&
698 Pieces of behaviour of a resource.\\
699 rr.interface\hfil\break
700 \makebox[0pt][l]{\scriptsize\ttfamily xpath:/capability/interface/}&
701 Information on access modes of a capability.\\
702 rr.intf\_param\hfil\break
703 \makebox[0pt][l]{\scriptsize\ttfamily xpath:/capability/interface/param/}&
704 Input parameters for services.\\
705 rr.relationship\hfil\break
706 \makebox[0pt][l]{\scriptsize\ttfamily xpath:/content/relationship/}&
707 Relationships between resources, e.g., mirroring, derivation, but
708 also providing access to data within a resource.\\
709 rr.res\_date\hfil\break
710 \makebox[0pt][l]{\scriptsize\ttfamily xpath:/curation/}&
711 A date associated with an event in the life cycle of the resource.
712 This could be creation or update. The role column can be used to
713 clarify.\\
714 rr.res\_detail\hfil\break
715 \makebox[0pt][l]{\scriptsize\ttfamily }&
716 XPath-value pairs for members of resource or capability and their
717 derivations that are less used and/or from VOResource extensions. The
718 pairs refer to a resource if cap\_index is NULL, to the referenced
719 capability otherwise.\\
720 rr.res\_role\hfil\break
721 \makebox[0pt][l]{\scriptsize\ttfamily }&
722 Entities, i.e., persons or organizations, operating on resources:
723 creators, contacts, publishers, contributors.\\
724 rr.res\_schema\hfil\break
725 \makebox[0pt][l]{\scriptsize\ttfamily xpath:/tableset/schema/}&
726 Sets of tables related to resources.\\
727 rr.res\_subject\hfil\break
728 \makebox[0pt][l]{\scriptsize\ttfamily xpath:/content/}&
729 Topics, object types, or other descriptive keywords about the
730 resource.\\
731 rr.res\_table\hfil\break
732 \makebox[0pt][l]{\scriptsize\ttfamily xpath:/(tableset/schema/|)table/}&
733 (Relational) tables that are part of schemata or resources.\\
734 rr.resource\hfil\break
735 \makebox[0pt][l]{\scriptsize\ttfamily xpath:/}&
736 The resources, i.e., services, data collections, organizations, etc.,
737 present in this registry.\\
738 rr.table\_column\hfil\break
739 \makebox[0pt][l]{\scriptsize\ttfamily xpath:/(tableset/schema/|)/table/column/}&
740 Metadata on columns of a resource's tables.\\
741 rr.validation\hfil\break
742 \makebox[0pt][l]{\scriptsize\ttfamily xpath:/(capability|)}&
743 Validation levels for resources and capabilities.\\
744
745 \sptablerule
746 \end{tabular}\hss}
747 \caption{The tables making up the TAP data model \texttt{Registry 1.0}}
748 \label{table:dm}
749 \end{table}
750
751 % /GENERATED
752
753 % HTML section start
754
755 \subsection{The resource Table}
756
757 \label{table_resource}
758
759 The \rtent{rr.resource} table contains most atomic members of
760 \rtent{vr:Resource} that have a 1:1 relationship to the resource
761 itself. Members of derived types are, in general, handled through
762 the \rtent{res\_detail}
763 table even if 1:1 (see \ref{table_res_detail}). The
764 \rtent{content\_level}, \rtent{content\_type}, \rtent{waveband},
765 and \rtent{rights} members are 1:n but still appear
766 here. If there are multiple values, they are concatenated with hash
767 characters (\#). Use the \rtent{ivo\_hashlist\_has} ADQL extension
768 function to check for the presence of a single value. This convention
769 saves on tables while not complicating common queries significantly.
770
771 A local addition is the \rtent{creator\_seq} column. It contains
772 all content of the \vorent{name} elements below a resource element
773 \vorent{curation} child's \vorent{creator} children, concatenated with a
774 sequence of semicolon and blank characters (``\mbox{\texttt{; }}''). The
775 individual parts must be concatenated preserving the sequence of the XML
776 elements. The resulting string is primarily intended for display
777 purposes (``author list'') and is hence not case-normalized. It was
778 added since the equivalent of an author list is expected to be a
779 metadatum that is displayed fairly frequently, but also since the
780 sequence of author names is generally considered significant. The
781 \rtent{res\_role} table, on the other hand, does not allow recovering
782 the input sequence of the rows belonging to one resource.
783
784 The \rtent{res\_type} column reflects the lower-cased value of
785 the \vorent{ri:Resource} element's \texttt{xsi:type} attribute,
786 where the canonical prefixes are used. While custom or experimental
787 VOResource extensions may lead to more or less arbitrary strings in that
788 column, VOResource and its IVOA-recommended extensions at the time of
789 writing define the following values for \rtent{res\_type}:
790
791
792 \begin{description}
793 \item[vg:authority]A naming authority (these records allow resolving who is responsible
794 for IVORNs with a certain authority; cf. \citet{std:VOID}).
795 \item[vg:registry]A registry is considered a publishing registry if it contains a
796 capability element with \texttt{xsi:type="vg:Harvest"}. Old,
797 RegistryInterface 1-compliant registries also use this type with a
798 capability of type \vorent{vg:Search}. The relational registry as
799 specified here, while superceding these old \vorent{vg:Search}
800 capabilities, does \emph{not} use this type any more. See section
801 \ref{registration} on how to locate services
802 supporting it.
803 \item[vr:organisation]The main purpose of an organisation as a registered resource is to
804 serve as a publisher of other resources.
805 \item[vr:resource]Any entity or component of a VO application that is describable and
806 identifiable by an IVOA identifier; while it is technically possible to
807 publish such records, the authors of such records should probably be
808 asked to use a more specific type.
809 \item[vr:service]A resource that can be invoked by a client to perform some action on
810 its behalf.
811 \item[vs:catalogservice]A service that interacts with one or more specified tables having
812 some coverage of the sky, time, and/or frequency.
813 \item[vs:dataservice]A service for accessing astronomical data; publishers choosing
814 this over \vorent{vs:catalogservice} probably intend to communicate
815 that there are no actual sky positions in the tables exposed.
816 \item[vs:datacollection]A schema as a logical grouping of data which, in general, is
817 composed of one or more accessible datasets. Use the
818 \rtent{rr.relationship} table to find out services that allow
819 access to the data (the \vorent{served\_by} relation), and/or look for values for
820 \vorent{/accessURL} in
821 \ref{table_res_detail}.
822 \item[vstd:standard]A description of a standard specification.
823
824 \end{description}
825
826 The \vorent{status} attribute of \vorent{vr:Resource} is
827 considered an implementation detail of the XML serialization and is not
828 kept here. Neither \vorent{inactive} nor \vorent{deleted}
829 records may be kept in the \rtent{resource} table. Since all
830 other tables in the relational registry should keep a foreign key on the
831 \rtent{ivoid} column, this implies that only metadata on
832 \vorent{active} records
833 is being kept in the relational registry. In other words, users can
834 expect a resource to exist and work if they find it in a relational
835 registry.
836
837
838 % GENERATED: maketable.sh rr.resource
839
840 \begin{inlinetable}
841 \small
842 \begin{tabular}{p{0.28\textwidth}p{0.2\textwidth}p{0.66\textwidth}}
843 \sptablerule
844 \multicolumn{3}{l}{\textit{Column names, utypes, ADQL types, and descriptions for the \rtent{rr.resource} table}}\\
845 \sptablerule
846 ivoid\hfil\break
847 \makebox[0pt][l]{\scriptsize\ttfamily xpath:identifier}&
848 \footnotesize VARCHAR(*)&
849 Unambiguous reference to the resource conforming to the IVOA standard for identifiers.\\
850 res\_type\hfil\break
851 \makebox[0pt][l]{\scriptsize\ttfamily xpath:@xsi:type}&
852 \footnotesize VARCHAR(*)&
853 Resource type (something like vs:datacollection, vs:catalogservice, etc).\\
854 created\hfil\break
855 \makebox[0pt][l]{\scriptsize\ttfamily xpath:@created}&
856 \footnotesize TIMESTAMP(1)&
857 The UTC date and time this resource metadata description was created. This timestamp must not be in the future. This time is not required to be accurate; it should be at least accurate to the day. Any insignificant time fields should be set to zero.\\
858 short\_name\hfil\break
859 \makebox[0pt][l]{\scriptsize\ttfamily xpath:shortName}&
860 \footnotesize VARCHAR(*)&
861 A short name or abbreviation given to something. This name will be used where brief annotations for the resource name are required. Applications may use it to refer to the resource in a compact display. One word or a few letters is recommended. No more than sixteen characters are allowed.\\
862 res\_title\hfil\break
863 \makebox[0pt][l]{\scriptsize\ttfamily xpath:title}&
864 \footnotesize VARCHAR(*)&
865 The full name given to the resource.\\
866 updated\hfil\break
867 \makebox[0pt][l]{\scriptsize\ttfamily xpath:@updated}&
868 \footnotesize TIMESTAMP(1)&
869 The UTC date this resource metadata description was last updated. This timestamp must not be in the future. This time is not required to be accurate; it should be at least accurate to the day. Any insignificant time fields should be set to zero.\\
870 content\_level\hfil\break
871 \makebox[0pt][l]{\scriptsize\ttfamily xpath:content/contentLevel}&
872 \footnotesize VARCHAR(*)&
873 A hash-separated list of content levels specifying the intended audience.\\
874 res\_description\hfil\break
875 \makebox[0pt][l]{\scriptsize\ttfamily xpath:content/description}&
876 \footnotesize VARCHAR(*)&
877 An account of the nature of the resource.\\
878 reference\_url\hfil\break
879 \makebox[0pt][l]{\scriptsize\ttfamily xpath:content/referenceURL}&
880 \footnotesize VARCHAR(*)&
881 URL pointing to a human-readable document describing this resource.\\
882 creator\_seq\hfil\break
883 \makebox[0pt][l]{\scriptsize\ttfamily xpath:curation/creator/name}&
884 \footnotesize VARCHAR(*)&
885 The creator(s) of the resource in the order given by the resource record author, separated by semicolons.\\
886 content\_type\hfil\break
887 \makebox[0pt][l]{\scriptsize\ttfamily xpath:content/type}&
888 \footnotesize VARCHAR(*)&
889 A hash-separated list of natures or genres of the content of the resource.\\
890 source\_format\hfil\break
891 \makebox[0pt][l]{\scriptsize\ttfamily xpath:content/source/@format}&
892 \footnotesize VARCHAR(*)&
893 The format of source\_value. Recognized values include "bibcode", referring to a standard astronomical bibcode (http://cdsweb.u-strasbg.fr/simbad/refcode.html).\\
894 source\_value\hfil\break
895 \makebox[0pt][l]{\scriptsize\ttfamily xpath:content/source}&
896 \footnotesize VARCHAR(*)&
897 A bibliographic reference from which the present resource is derived or extracted.\\
898 res\_version\hfil\break
899 \makebox[0pt][l]{\scriptsize\ttfamily xpath:curation/version}&
900 \footnotesize VARCHAR(*)&
901 Label associated with creation or availablilty of a version of a resource.\\
902 region\_of\_regard\hfil\break
903 \makebox[0pt][l]{\scriptsize\ttfamily xpath:coverage/regionOfRegard}&
904 \footnotesize REAL(1)&
905 A single numeric value representing the angle, given in decimal degrees, by which a positional query against this resource should be "blurred" in order to get an appropriate match.\\
906 waveband\hfil\break
907 \makebox[0pt][l]{\scriptsize\ttfamily xpath:coverage/waveband}&
908 \footnotesize VARCHAR(*)&
909 A hash-separated list of regions of the electro-magnetic spectrum that the resource's spectral coverage overlaps with.\\
910 rights\hfil\break
911 \makebox[0pt][l]{\scriptsize\ttfamily xpath:rights}&
912 \footnotesize VARCHAR(*)&
913 Information about rights held in and over the resource (multiple values are separated by hashes).\\
914
915 \sptablerule
916 \end{tabular}
917 \end{inlinetable}
918
919
920 % /GENERATED
921
922
923 This table should have the \rtent{ivoid} column explicitly set
924 as its primary key.
925
926 The following columns MUST be lowercased during ingestion:
927 \rtent{ivoid}, \rtent{res\_type}, \rtent{content\_level},
928 \rtent{content\_type}, \rtent{source\_format},
929 \rtent{waveband}.
930 Clients are advised to query the \rtent{res\_description} and
931 \rtent{res\_title} columns
932 using the the \rtent{ivo\_hasword} function, and to use
933 \rtent{ivo\_hashlist\_has} on \rtent{content\_level},
934 \rtent{content\_type},
935 \rtent{waveband}, and \rtent{rights}.
936
937 The row for \rtent{region\_of\_regard} in
938 \tapent{TAP\_SCHEMA.columns} MUST have \texttt{deg} in its
939 \tapent{unit} column.
940
941
942 % HTML section ends
943
944 % subsection table\_resource
945 % HTML section start
946
947 \subsection{The res\_role Table}
948
949 \label{table_res_role}
950
951 This table subsumes the contact, publisher, contributor,
952 and creator members of the
953 VOResource data model. They have been combined into a single table to
954 reduce the total number of tables, and also in anticipation of a unified
955 data model for such entities in future versions of VOResource.
956
957 The actual role is given in the \rtent{base\_role} column, which
958 can be one of \rtent{contact}, \rtent{publisher}, \rtent{contributor}, or
959 \rtent{creator}. Depending on this value, here are the xpaths
960 for the table fields (we have abbreviated
961 \vorent{/curation/publisher}
962 as cp, \vorent{/curation/contact} as co, \vorent{/curation/creator}
963 as cc,
964 and \vorent{/curation/contributor} as cb):
965
966 \vspace{5pt}
967 \hbox to\hsize{\hss\small
968 \noindent\begin{tabular}{lllll}
969 \sptablerule
970 \textbf{base\_role value}&
971 \textbf{contact}&
972 \textbf{publisher}&
973 \textbf{creator}&
974 \textbf{contributor}\\
975 \sptablerule
976 role\_name&co/name&cp&cc/name&cb\\
977 role\_ivoid&co/name/@ivo-id&cp/@ivo-id&cc/name/@ivo-id&cb/@ivo-id\\
978 address&co/address&N/A&N/A&N/A\\
979 email&co/email&N/A&N/A&N/A\\
980 telephone&co/telephone&N/A&N/A&N/A\\
981 logo&co/logo&N/A&cc/logo&N/A\\
982 \sptablerule
983 \end{tabular}\hss}
984 \vskip5pt
985
986 Not all columns are available for each role type in VOResource. For
987 example, contacts have no logo, and creators no telephone members. Unavailable
988 metadata (marked with N/A in the above table) MUST be represented with NULL
989 values in the corresponding columns.
990
991 Note that, due to current practice in the VO, it is not easy to
992 predict what \rtent{role\_name} will contain; it could be a single
993 name, where again the actual format is unpredictable
994 (full first name, initials in front or behind, or
995 even a project name), but it could as well be a full author list. Thus,
996 when matching against \rtent{role\_name}, you will have to use
997 rather lenient regular expressions. Changing this, admittedly
998 regrettable, situation would
999 probably require a change in the VOResource schema.
1000
1001
1002 % GENERATED: maketable.sh rr.res_role
1003
1004 \begin{inlinetable}
1005 \small
1006 \begin{tabular}{p{0.28\textwidth}p{0.2\textwidth}p{0.66\textwidth}}
1007 \sptablerule
1008 \multicolumn{3}{l}{\textit{Column names, utypes, ADQL types, and descriptions for the \rtent{rr.res\_role} table}}\\
1009 \sptablerule
1010 ivoid\hfil\break
1011 \makebox[0pt][l]{\scriptsize\ttfamily xpath:/identifier}&
1012 \footnotesize VARCHAR(*)&
1013 The parent resource.\\
1014 role\_name\hfil\break
1015 \makebox[0pt][l]{\scriptsize\ttfamily }&
1016 \footnotesize VARCHAR(*)&
1017 The real-world name or title of a person or organization.\\
1018 role\_ivoid\hfil\break
1019 \makebox[0pt][l]{\scriptsize\ttfamily }&
1020 \footnotesize VARCHAR(*)&
1021 An IVOA identifier of a person or organization.\\
1022 street\_address\hfil\break
1023 \makebox[0pt][l]{\scriptsize\ttfamily }&
1024 \footnotesize VARCHAR(*)&
1025 A mailing address for a person or organization.\\
1026 email\hfil\break
1027 \makebox[0pt][l]{\scriptsize\ttfamily }&
1028 \footnotesize VARCHAR(*)&
1029 An email address the entity can be reached at.\\
1030 telephone\hfil\break
1031 \makebox[0pt][l]{\scriptsize\ttfamily }&
1032 \footnotesize VARCHAR(*)&
1033 A telephone number the entity can be reached at.\\
1034 logo\hfil\break
1035 \makebox[0pt][l]{\scriptsize\ttfamily }&
1036 \footnotesize VARCHAR(*)&
1037 URL pointing to a graphical logo, which may be used to help identify the entity.\\
1038 base\_role\hfil\break
1039 \makebox[0pt][l]{\scriptsize\ttfamily }&
1040 \footnotesize VARCHAR(*)&
1041 The role played by this entity; this is one of contact, publisher, and creator.\\
1042
1043 \sptablerule
1044 \end{tabular}
1045 \end{inlinetable}
1046
1047
1048 % /GENERATED
1049
1050
1051
1052 The \rtent{ivoid} column should be an explicit foreign key into
1053 the \rtent{resource} table. It is recommended to maintain indexes
1054 on at least the \rtent{role\_name} column, ideally in a way that
1055 supports regular expressions.
1056
1057 The following columns MUST be lowercased during ingestion:
1058 \rtent{ivoid}, \rtent{role\_ivoid},
1059 \rtent{base\_role}.
1060 Clients are advised to query the remaining columns, in particular
1061 \rtent{role\_name},
1062 case-insensitively, e.g., using \rtent{ivo\_nocasematch}.
1063
1064
1065 % HTML section ends
1066
1067 % subsection table\_res\_role
1068 % HTML section start
1069
1070 \subsection{The res\_subject Table}
1071
1072 \label{table_res_subject}
1073
1074 Since subject queries are expected to be frequent and perform relatively
1075 complex checks (e.g., resulting from thesaurus queries in the clients), the
1076 subjects are kept in a separate table rather than being hash-joined like other
1077 string-like 1:n members of resource.
1078
1079
1080 % GENERATED: maketable.sh rr.res_subject
1081
1082 \begin{inlinetable}
1083 \small
1084 \begin{tabular}{p{0.28\textwidth}p{0.2\textwidth}p{0.66\textwidth}}
1085 \sptablerule
1086 \multicolumn{3}{l}{\textit{Column names, utypes, ADQL types, and descriptions for the \rtent{rr.res\_subject} table}}\\
1087 \sptablerule
1088 ivoid\hfil\break
1089 \makebox[0pt][l]{\scriptsize\ttfamily xpath:/identifier}&
1090 \footnotesize VARCHAR(*)&
1091 The parent resource.\\
1092 res\_subject\hfil\break
1093 \makebox[0pt][l]{\scriptsize\ttfamily xpath:subject}&
1094 \footnotesize VARCHAR(*)&
1095 Topics, object types, or other descriptive keywords about the resource.\\
1096
1097 \sptablerule
1098 \end{tabular}
1099 \end{inlinetable}
1100
1101
1102 % /GENERATED
1103
1104
1105
1106 The \rtent{ivoid} column should be an explicit foreign key into
1107 \rtent{resource}. It is recommended to index the
1108 \rtent{res\_subject} column, preferably in a way that allows to process
1109 case-insensitive and pattern queries using the index.
1110
1111 The \rtent{ivoid} column MUST be lowercased during
1112 ingestion. Clients are advised to query the \rtent{res\_subject} column
1113 case-insensitively, e.g., using \rtent{ivo\_nocasematch}.
1114
1115
1116 % HTML section ends
1117
1118 % subsection table\_subject
1119 % HTML section start
1120
1121 \subsection{The capability Table}
1122
1123 \label{table_capability}
1124
1125 The capability table describes a resource's modes of interaction; it only
1126 contains the members of the base type \vorent{vr:Capability}.
1127 Members of derived types are kept in the \rtent{res\_detail} table
1128 (see \ref{table_res_detail}).
1129
1130 The table has a
1131 \rtent{cap\_index} to disambiguate multiple
1132 capabilities on a single resource. See section \ref{primarykeys} for details.
1133
1134
1135 % GENERATED: maketable.sh rr.capability
1136
1137 \begin{inlinetable}
1138 \small
1139 \begin{tabular}{p{0.28\textwidth}p{0.2\textwidth}p{0.66\textwidth}}
1140 \sptablerule
1141 \multicolumn{3}{l}{\textit{Column names, utypes, ADQL types, and descriptions for the \rtent{rr.capability} table}}\\
1142 \sptablerule
1143 ivoid\hfil\break
1144 \makebox[0pt][l]{\scriptsize\ttfamily xpath:/identifier}&
1145 \footnotesize VARCHAR(*)&
1146 The parent resource.\\
1147 cap\_index\hfil\break
1148 \makebox[0pt][l]{\scriptsize\ttfamily }&
1149 \footnotesize SMALLINT(1)&
1150 An arbitrary identifier of this capability within the resource.\\
1151 cap\_type\hfil\break
1152 \makebox[0pt][l]{\scriptsize\ttfamily xpath:@xsi:type}&
1153 \footnotesize VARCHAR(*)&
1154 The type of capability covered here.\\
1155 cap\_description\hfil\break
1156 \makebox[0pt][l]{\scriptsize\ttfamily xpath:description}&
1157 \footnotesize VARCHAR(*)&
1158 A human-readable description of what this capability provides as part of the over-all service.\\
1159 standard\_id\hfil\break
1160 \makebox[0pt][l]{\scriptsize\ttfamily xpath:@standardID}&
1161 \footnotesize VARCHAR(*)&
1162 A URI for a standard this capability conforms to.\\
1163
1164 \sptablerule
1165 \end{tabular}
1166 \end{inlinetable}
1167
1168
1169 % /GENERATED
1170
1171
1172
1173 This table should have an explicit primary key made up of
1174 \rtent{ivoid} and \rtent{cap\_index}.
1175 The \rtent{ivoid} column should be
1176 an explicit foreign key into \rtent{resource}.
1177 It is recommended to maintain indexes on at least the
1178 \rtent{cap\_type} and \rtent{standard\_id} columns.
1179
1180 The following columns MUST be lowercased during ingestion:
1181 \rtent{ivoid}, \rtent{cap\_type}, \rtent{standard\_id}.
1182 Clients are advised to query the \rtent{cap\_description} column
1183 using the \rtent{ivo\_hasword} function.
1184
1185
1186 % HTML section ends
1187
1188 % subsection table\_capability
1189 % HTML section start
1190
1191 \subsection{The res\_schema Table}
1192
1193 \label{table_res_schema}
1194
1195 The \rtent{res\_schema} table corresponds to VODataService's
1196 \vorent{schema} element. It has been renamed to avoid clashes with
1197 the SQL reserved word \texttt{SCHEMA}.
1198
1199 The table has a column \rtent{schema\_index} to disambiguate
1200 multiple schema elements on a single resource. See section \ref{primarykeys} for details.
1201
1202
1203 % GENERATED: maketable.sh rr.res_schema
1204
1205 \begin{inlinetable}
1206 \small
1207 \begin{tabular}{p{0.28\textwidth}p{0.2\textwidth}p{0.66\textwidth}}
1208 \sptablerule
1209 \multicolumn{3}{l}{\textit{Column names, utypes, ADQL types, and descriptions for the \rtent{rr.res\_schema} table}}\\
1210 \sptablerule
1211 ivoid\hfil\break
1212 \makebox[0pt][l]{\scriptsize\ttfamily xpath:/identifier}&
1213 \footnotesize VARCHAR(*)&
1214 The parent resource.\\
1215 schema\_index\hfil\break
1216 \makebox[0pt][l]{\scriptsize\ttfamily }&
1217 \footnotesize SMALLINT(1)&
1218 An arbitrary identifier for the res\_schema rows belonging to a resource.\\
1219 schema\_description\hfil\break
1220 \makebox[0pt][l]{\scriptsize\ttfamily xpath:description}&
1221 \footnotesize VARCHAR(*)&
1222 A free text description of the tableset explaining in general how all of the tables are related.\\
1223 schema\_name\hfil\break
1224 \makebox[0pt][l]{\scriptsize\ttfamily xpath:name }&
1225 \footnotesize VARCHAR(*)&
1226 A name for the set of tables.\\
1227 schema\_title\hfil\break
1228 \makebox[0pt][l]{\scriptsize\ttfamily xpath:title}&
1229 \footnotesize VARCHAR(*)&
1230 A descriptive, human-interpretable name for the table set.\\
1231 schema\_utype\hfil\break
1232 \makebox[0pt][l]{\scriptsize\ttfamily xpath:utype}&
1233 \footnotesize VARCHAR(*)&
1234 An identifier for a concept in a data model that the data in this schema as a whole represent.\\
1235
1236 \sptablerule
1237 \end{tabular}
1238 \end{inlinetable}
1239
1240
1241 % /GENERATED
1242
1243
1244
1245 This table should have an explicit primary key made up of
1246 \rtent{ivoid} and \rtent{schema\_index}. The
1247 \rtent{ivoid} column should be an explicit foreign key into
1248 \rtent{resource}.
1249
1250 The following columns MUST be lowercased during ingestion:
1251 \rtent{ivoid}, \rtent{schema\_name}, \rtent{schema\_utype}.
1252 Clients are advised to query the \rtent{schema\_description}
1253 and \rtent{schema\_title} columns
1254 using the the \rtent{ivo\_hasword} function.
1255
1256
1257 % HTML section ends
1258
1259 % subsection table\_res\_schema
1260 % HTML section start
1261
1262 \subsection{The res\_table Table}
1263
1264 \label{table_res_table}
1265
1266 The \rtent{res\_table} table models VODataService's
1267 \vorent{table} element. It has been renamed to avoid name clashes
1268 with the SQL reserved word \texttt{TABLE}.
1269
1270 VODataService 1.0 had a similar element that was a direct child of
1271 resource. Ingestors should also accept such tables, as there are still
1272 numerous active VODataService 1.0 resources in the Registry at the time
1273 of writing (this is the reason for the alternative in the table xpath).
1274
1275 The table contains a column \rtent{table\_index} to disambiguate
1276 multiple tables on a single resource. See section \ref{primarykeys} for details. Note that if the sibling
1277 count is used as implementation of \rtent{table\_index}, the count
1278 must be per resource and \emph{not} per schema, as
1279 \rtent{table\_index} MUST be unique within a resource.
1280
1281
1282 % GENERATED: maketable.sh rr.res_table
1283
1284 \begin{inlinetable}
1285 \small
1286 \begin{tabular}{p{0.28\textwidth}p{0.2\textwidth}p{0.66\textwidth}}
1287 \sptablerule
1288 \multicolumn{3}{l}{\textit{Column names, utypes, ADQL types, and descriptions for the \rtent{rr.res\_table} table}}\\
1289 \sptablerule
1290 ivoid\hfil\break
1291 \makebox[0pt][l]{\scriptsize\ttfamily xpath:/identifier}&
1292 \footnotesize VARCHAR(*)&
1293 The parent resource.\\
1294 schema\_index\hfil\break
1295 \makebox[0pt][l]{\scriptsize\ttfamily }&
1296 \footnotesize SMALLINT(1)&
1297 Index of the schema this table belongs to, if it belongs to a schema (otherwise NULL).\\
1298 table\_description\hfil\break
1299 \makebox[0pt][l]{\scriptsize\ttfamily xpath:description}&
1300 \footnotesize VARCHAR(*)&
1301 A free-text description of the table's contents.\\
1302 table\_name\hfil\break
1303 \makebox[0pt][l]{\scriptsize\ttfamily xpath:name}&
1304 \footnotesize VARCHAR(*)&
1305 The fully qualified name of the table. This name should include all catalog or schema prefixes needed to distinguish it in a query.\\
1306 table\_index\hfil\break
1307 \makebox[0pt][l]{\scriptsize\ttfamily }&
1308 \footnotesize SMALLINT(1)&
1309 An arbitrary identifier for the tables belonging to a resource.\\
1310 table\_title\hfil\break
1311 \makebox[0pt][l]{\scriptsize\ttfamily xpath:title}&
1312 \footnotesize VARCHAR(*)&
1313 A descriptive, human-interpretable name for the table.\\
1314 table\_type\hfil\break
1315 \makebox[0pt][l]{\scriptsize\ttfamily xpath:@type}&
1316 \footnotesize VARCHAR(*)&
1317 A name for the role this table plays. Recognized values include "output", indicating this table is output from a query; "base\_table", indicating a table whose records represent the main subjects of its schema; and "view", indicating that the table represents a useful combination or subset of other tables. Other values are allowed.\\
1318 table\_utype\hfil\break
1319 \makebox[0pt][l]{\scriptsize\ttfamily xpath:utype}&
1320 \footnotesize VARCHAR(*)&
1321 An identifier for a concept in a data model that the data in this table as a whole represent.\\
1322
1323 \sptablerule
1324 \end{tabular}
1325 \end{inlinetable}
1326
1327
1328 % /GENERATED
1329
1330
1331
1332 This table should have an explicit primary key made up of
1333 \rtent{ivoid} and \rtent{table\_index}. The
1334 \rtent{ivoid} column should be an explicit
1335 foreign key into \rtent{resource}. It is recommended to
1336 maintain an index on at least the \rtent{table\_description}
1337 column, ideally one suited for queries with \rtent{ivo\_hasword}.
1338
1339 The following columns MUST be lowercased during ingestion:
1340 \rtent{ivoid}, \rtent{table\_name}, \rtent{table\_type},
1341 \rtent{table\_utype}.
1342 Clients are advised to query the \rtent{table\_description}
1343 and \rtent{table\_title} columns
1344 using the the \rtent{ivo\_hasword} function.
1345
1346
1347 % HTML section ends
1348
1349 % subsection table\_res\_table
1350 % HTML section start
1351
1352 \subsection{The table\_column Table}
1353
1354 \label{table_table_column}
1355
1356 The \rtent{table\_column} table models the content of VODataService's
1357 \vorent{column} element. The table has been renamed to avoid
1358 a name clash with the SQL reserved word \texttt{COLUMN}.
1359
1360 Since it is expected that queries for column properties will be
1361 fairly common in advanced queries, it is the column table that has the
1362 unprefixed versions of common member names (name, ucd,
1363 utype, etc).
1364
1365 The \rtent{flag} column contains a concatenation of all values
1366 of a \vorent{column} element's \vorent{flag} children, separated
1367 by hash characters. Use the \rtent{ivo\_hashlist\_has} function in
1368 queries against \rtent{flag}.
1369
1370 The \rtent{table\_column} table also includes information from
1371 VODataService's data type concept. VODataService 1.1 includes several type
1372 systems (VOTable, ADQL, Simple). The
1373 \rtent{type\_system} column contains the value of the column's
1374 \vorent{datatype} child, with the VODataService XML prefix fixed
1375 to vs; hence, this column will contain one of NULL,
1376 \texttt{vs:taptype},
1377 \texttt{vs:simpledatatype}, and \texttt{vs:votabletype}.
1378
1379
1380 % GENERATED: maketable.sh rr.table_column
1381
1382 \begin{inlinetable}
1383 \small
1384 \begin{tabular}{p{0.28\textwidth}p{0.2\textwidth}p{0.66\textwidth}}
1385 \sptablerule
1386 \multicolumn{3}{l}{\textit{Column names, utypes, ADQL types, and descriptions for the \rtent{rr.table\_column} table}}\\
1387 \sptablerule
1388 ivoid\hfil\break
1389 \makebox[0pt][l]{\scriptsize\ttfamily xpath:/identifier}&
1390 \footnotesize VARCHAR(*)&
1391 The parent resource.\\
1392 table\_index\hfil\break
1393 \makebox[0pt][l]{\scriptsize\ttfamily }&
1394 \footnotesize SMALLINT(1)&
1395 Index of the table this column belongs to.\\
1396 name\hfil\break
1397 \makebox[0pt][l]{\scriptsize\ttfamily xpath:name}&
1398 \footnotesize VARCHAR(*)&
1399 The name of the column.\\
1400 ucd\hfil\break
1401 \makebox[0pt][l]{\scriptsize\ttfamily xpath:ucd}&
1402 \footnotesize VARCHAR(*)&
1403 A unified content descriptor that describes the scientific content of the parameter.\\
1404 unit\hfil\break
1405 \makebox[0pt][l]{\scriptsize\ttfamily xpath:unit}&
1406 \footnotesize VARCHAR(*)&
1407 The unit associated with all values in the column.\\
1408 utype\hfil\break
1409 \makebox[0pt][l]{\scriptsize\ttfamily xpath:utype}&
1410 \footnotesize VARCHAR(*)&
1411 An identifier for a role in a data model that the data in this column represents.\\
1412 std\hfil\break
1413 \makebox[0pt][l]{\scriptsize\ttfamily xpath:@std}&
1414 \footnotesize SMALLINT(1)&
1415 If 1, the meaning and use of this parameter is reserved and defined by a standard model. If 0, it represents a database-specific parameter that effectively extends beyond the standard.\\
1416 datatype\hfil\break
1417 \makebox[0pt][l]{\scriptsize\ttfamily xpath:dataType}&
1418 \footnotesize VARCHAR(*)&
1419 The type of the data contained in the column.\\
1420 extended\_schema\hfil\break
1421 \makebox[0pt][l]{\scriptsize\ttfamily xpath:dataType/@extendedSchema}&
1422 \footnotesize VARCHAR(*)&
1423 An identifier for the schema that the value given by the extended attribute is drawn from.\\
1424 extended\_type\hfil\break
1425 \makebox[0pt][l]{\scriptsize\ttfamily xpath:dataType/@extendedType}&
1426 \footnotesize VARCHAR(*)&
1427 A custom type for the values this column contains.\\
1428 arraysize\hfil\break
1429 \makebox[0pt][l]{\scriptsize\ttfamily xpath:dataType/@arraysize}&
1430 \footnotesize VARCHAR(*)&
1431 The shape of the array that constitutes the value, e.g., 4, *, 4*, 5x4, or 5x*, as specified by VOTable.\\
1432 delim\hfil\break
1433 \makebox[0pt][l]{\scriptsize\ttfamily xpath:dataType/@delim}&
1434 \footnotesize VARCHAR(*)&
1435 The string that is used to delimit elements of an array value when arraysize is not '1'.\\
1436 type\_system\hfil\break
1437 \makebox[0pt][l]{\scriptsize\ttfamily xpath:dataType/@xsi:type}&
1438 \footnotesize VARCHAR(*)&
1439 The type system used, as a QName with a canonical prefix; this will ususally be one of vs:simpledatatype, vs:votabletype, and vs:taptype.\\
1440 flag\hfil\break
1441 \makebox[0pt][l]{\scriptsize\ttfamily xpath:flag}&
1442 \footnotesize VARCHAR(*)&
1443 Hash-separated keywords representing traits of the column. Recognized values include "indexed", "primary", and "nullable".\\
1444 column\_description\hfil\break
1445 \makebox[0pt][l]{\scriptsize\ttfamily xpath:description}&
1446 \footnotesize VARCHAR(*)&
1447 A free-text description of the column's contents.\\
1448
1449 \sptablerule
1450 \end{tabular}
1451 \end{inlinetable}
1452
1453
1454 % /GENERATED
1455
1456
1457
1458 The pair \rtent{ivoid}, \rtent{table\_index} should be an
1459 explicit foreign key into \rtent{res\_table}. It is recommended to
1460 maintain indexes on at least the \rtent{column\_description},
1461 \rtent{name}, \rtent{ucd}, and \rtent{utype} columns,
1462 where the index on \rtent{column\_description} should ideally be able
1463 to handle queries using \rtent{ivo\_hasword}.
1464
1465 The following columns MUST be lowercased during ingestion:
1466 \rtent{ivoid}, \rtent{name}, \rtent{ucd},
1467 \rtent{utype}, \rtent{datatype}, \rtent{type\_system}.
1468 The boolean value of the column's \rtent{std} attribute must be
1469 converted to 0 (False), 1 (True), or NULL (not given) on ingestion.
1470 Clients are advised to query the \rtent{description}
1471 column using the \rtent{ivo\_hasword} function, and to query
1472 the \rtent{flag} column using the \rtent{ivo\_hashlist\_has}
1473 function.
1474
1475
1476 % HTML section ends
1477
1478 % subsection table\_table\_column
1479 % HTML section start
1480
1481 \subsection{The interface Table}
1482
1483 \label{table_interface}
1484
1485 The \rtent{interface} table subsumes both the
1486 \vorent{vr:Interface} and \vorent{vr:access\-URL} types from
1487 VOResource. The integration of \vorent{access\-URL} into
1488 the \rtent{interface} table means that an interface in the
1489 relational registry can only have one access URL, where in VOResource it
1490 can have many. In practice, this particular VOResource capability has
1491 not been used by registry record authors. Since access URLs are
1492 probably the item most queried for, it seems warranted to save one
1493 indirection when querying for them.
1494
1495 This specification deprecates the \texttt{maxOccurs=\dquote unbounded"}
1496 in the definition of \vorent{Interface}'s \vorent{accessURL}
1497 child in the XML schema
1498 \texttt{http://www.ivoa.net/xml/VOResource/v1.0}; in future versions
1499 of VOResource, implementations can expect this to be
1500 \texttt{maxOccurs="1"}. Meanwhile, implementation behavior in the
1501 presence of multiple access URLs in an interface is undefined.
1502
1503 The table contains a column \rtent{intf\_index} to disambiguate
1504 multiple interfaces of one resource. See section \ref{primarykeys} for details.
1505
1506 In VOResource, interfaces can have zero or more \vorent{securityMethod}
1507 children to convey support for authentication and authorization methods. At
1508 the time of writing, only a \vorent{standardId} attribute is defined on
1509 these, and the Registry only contains a single resource record using
1510 \vorent{securityMethod}. It was therefore decided to map this information
1511 into the \rtent{res\_detail} table using the detail xpath
1512 \texttt{/capability/interface/securityMethod/@standardID}, with
1513 \rtent{cap\_index} set so the embedding capability is referenced;
1514 this implies that RegTAP does not allow distinguishing between
1515 interfaces for security methods. If \vorent{securityMethod} will
1516 come into common use as the VO evolves, this design will be reconsidered
1517 if reliable RegTAP-based discovery of access URLs in multi-interface
1518 multi-authentication scenarios is found necessary.
1519
1520 The \rtent{query\_type} column is a hash-joined list (analogous
1521 to \rtent{waveband}, etc. in the resource table), as
1522 the XML schema allows listing up to two request methods.
1523
1524 This table only contains interface elements from within capabilities.
1525 Interface elements in StandardsRegExt records are ignored in the
1526 relational registry,
1527 and they must not be inserted in this table, since doing so would disturb
1528 the foreign key from interface into capability. In other words,
1529 the relational registry requires every interface to have a parent capability.
1530
1531 Analogous to \rtent{resource.res\_type}, the
1532 \rtent{intf\_type} column contains type names; VOResource extensions
1533 can define new types here, but at the time of writing, the following
1534 types are mentioned in IVOA-recommended schemata:
1535
1536
1537 \begin{description}
1538 \item[vs:paramhttp]A service invoked via an HTTP query, usually with some form of
1539 structured parameters. This type is used for interfaces speaking
1540 standard IVOA protocols.
1541 \item[vr:webbrowser]A (form-based) interface intended to be accessed interactively by a
1542 user via a web browser.
1543 \item[vg:oaihttp]A standard OAI PMH interface using HTTP queries with form-urlencoded
1544 parameters.
1545 \item[vg:oaisoap]A standard OAI PMH interface using a SOAP Web Service
1546 interface.
1547 \item[vr:webservice]A Web Service that is describable by a WSDL document.
1548
1549 \end{description}
1550
1551
1552 % GENERATED: maketable.sh rr.interface
1553
1554 \begin{inlinetable}
1555 \small
1556 \begin{tabular}{p{0.28\textwidth}p{0.2\textwidth}p{0.66\textwidth}}
1557 \sptablerule
1558 \multicolumn{3}{l}{\textit{Column names, utypes, ADQL types, and descriptions for the \rtent{rr.interface} table}}\\
1559 \sptablerule
1560 ivoid\hfil\break
1561 \makebox[0pt][l]{\scriptsize\ttfamily xpath:/identifier}&
1562 \footnotesize VARCHAR(*)&
1563 The parent resource.\\
1564 cap\_index\hfil\break
1565 \makebox[0pt][l]{\scriptsize\ttfamily }&
1566 \footnotesize SMALLINT(1)&
1567 The index of the parent capability.\\
1568 intf\_index\hfil\break
1569 \makebox[0pt][l]{\scriptsize\ttfamily }&
1570 \footnotesize SMALLINT(1)&
1571 An arbitrary identifier for the interfaces of a resource.\\
1572 intf\_type\hfil\break
1573 \makebox[0pt][l]{\scriptsize\ttfamily xpath:@xsi:type}&
1574 \footnotesize VARCHAR(*)&
1575 The type of the interface (vr:webbrowser, vs:paramhttp, etc).\\
1576 intf\_role\hfil\break
1577 \makebox[0pt][l]{\scriptsize\ttfamily xpath:@role}&
1578 \footnotesize VARCHAR(*)&
1579 An identifier for the role the interface plays in the particular capability. If the value is equal to "std" or begins with "std:", then the interface refers to a standard interface defined by the standard referred to by the capability's standardID attribute.\\
1580 std\_version\hfil\break
1581 \makebox[0pt][l]{\scriptsize\ttfamily xpath:@version}&
1582 \footnotesize VARCHAR(*)&
1583 The version of a standard interface specification that this interface complies with. When the interface is provided in the context of a Capability element, then the standard being refered to is the one identified by the Capability's standardID element.\\
1584 query\_type\hfil\break
1585 \makebox[0pt][l]{\scriptsize\ttfamily xpath:queryType}&
1586 \footnotesize VARCHAR(*)&
1587 Hash-joined list of expected HTTP method (get or post) supported by the service.\\
1588 result\_type\hfil\break
1589 \makebox[0pt][l]{\scriptsize\ttfamily xpath:resultType}&
1590 \footnotesize VARCHAR(*)&
1591 The MIME type of a document returned in the HTTP response.\\
1592 wsdl\_url\hfil\break
1593 \makebox[0pt][l]{\scriptsize\ttfamily xpath:wsdlURL}&
1594 \footnotesize VARCHAR(*)&
1595 The location of the WSDL that describes this Web Service. If NULL, the location can be assumed to be the accessURL with '?wsdl' appended.\\
1596 url\_use\hfil\break
1597 \makebox[0pt][l]{\scriptsize\ttfamily xpath:accessURL/@use}&
1598 \footnotesize VARCHAR(*)&
1599 A flag indicating whether this should be interpreted as a base URL ('base'), a full URL ('full'), or a URL to a directory that will produce a listing of files ('dir').\\
1600 access\_url\hfil\break
1601 \makebox[0pt][l]{\scriptsize\ttfamily xpath:accessURL}&
1602 \footnotesize VARCHAR(*)&
1603 The URL at which the interface is found.\\
1604
1605 \sptablerule
1606 \end{tabular}
1607 \end{inlinetable}
1608
1609
1610 % /GENERATED
1611
1612 This table should have the pair \rtent{ivoid}, \rtent{cap\_index}
1613 as an explicit foreign key into
1614 \rtent{capability}, and the pair \rtent{ivoid}, and
1615 \rtent{intf\_index} as an explicit primary key. Additionally, it
1616 is recommended to maintain an index on at least the
1617 \rtent{intf\_type} column.
1618
1619 The following columns MUST be lowercased during ingestion:
1620 \rtent{ivoid}, \rtent{intf\_type}, \rtent{intf\_role},
1621 \rtent{std\_version}, \rtent{query\_type},
1622 \rtent{result\_type}, \rtent{url\_use}.
1623 Clients are advised to query \rtent{query\_type} using the the
1624 \rtent{ivo\_hashlist\_has} function.
1625
1626
1627 % HTML section ends
1628
1629 % subsection table\_interface
1630 % HTML section start
1631
1632 \subsection{The intf\_param Table}
1633
1634 \label{table_intf_param}
1635
1636 The \rtent{intf\_param} table keeps information on the parameters
1637 available on interfaces. It is therefore closely related to
1638 \rtent{table\_column}, but the differences between the two are
1639 significant enough to warrant a separation between the two tables.
1640 Since the names of common column attributes are used where applicable in
1641 both tables (e.g., name, ucd, etc), the two tables cannot be (naturally)
1642 joined.
1643
1644
1645 % GENERATED: maketable.sh rr.intf_param
1646
1647 \begin{inlinetable}
1648 \small
1649 \begin{tabular}{p{0.28\textwidth}p{0.2\textwidth}p{0.66\textwidth}}
1650 \sptablerule
1651 \multicolumn{3}{l}{\textit{Column names, utypes, ADQL types, and descriptions for the \rtent{rr.intf\_param} table}}\\
1652 \sptablerule
1653 ivoid\hfil\break
1654 \makebox[0pt][l]{\scriptsize\ttfamily xpath:/identifier}&
1655 \footnotesize VARCHAR(*)&
1656 The parent resource.\\
1657 intf\_index\hfil\break
1658 \makebox[0pt][l]{\scriptsize\ttfamily }&
1659 \footnotesize SMALLINT(1)&
1660 The index of the interface this parameter belongs to.\\
1661 name\hfil\break
1662 \makebox[0pt][l]{\scriptsize\ttfamily xpath:name}&
1663 \footnotesize VARCHAR(*)&
1664 The name of the parameter.\\
1665 ucd\hfil\break
1666 \makebox[0pt][l]{\scriptsize\ttfamily xpath:ucd}&
1667 \footnotesize VARCHAR(*)&
1668 A unified content descriptor that describes the scientific content of the parameter.\\
1669 unit\hfil\break
1670 \makebox[0pt][l]{\scriptsize\ttfamily xpath:unit}&
1671 \footnotesize VARCHAR(*)&
1672 The unit associated with all values in the parameter.\\
1673 utype\hfil\break
1674 \makebox[0pt][l]{\scriptsize\ttfamily xpath:utype}&
1675 \footnotesize VARCHAR(*)&
1676 An identifier for a role in a data model that the data in this parameter represents.\\
1677 std\hfil\break
1678 \makebox[0pt][l]{\scriptsize\ttfamily xpath:@std}&
1679 \footnotesize SMALLINT(1)&
1680 If 1, the meaning and use of this parameter is reserved and defined by a standard model. If 0, it represents a database-specific parameter that effectively extends beyond the standard.\\
1681 datatype\hfil\break
1682 \makebox[0pt][l]{\scriptsize\ttfamily xpath:dataType}&
1683 \footnotesize VARCHAR(*)&
1684 The type of the data contained in the parameter.\\
1685 extended\_schema\hfil\break
1686 \makebox[0pt][l]{\scriptsize\ttfamily xpath:dataType/@extendedSchema}&
1687 \footnotesize VARCHAR(*)&
1688 An identifier for the schema that the value given by the extended attribute is drawn from.\\
1689 extended\_type\hfil\break
1690 \makebox[0pt][l]{\scriptsize\ttfamily xpath:dataType/@extendedType}&
1691 \footnotesize VARCHAR(*)&
1692 A custom type for the values this parameter contains.\\
1693 arraysize\hfil\break
1694 \makebox[0pt][l]{\scriptsize\ttfamily xpath:dataType/@arraysize}&
1695 \footnotesize VARCHAR(*)&
1696 The shape of the array that constitutes the value, e.g., 4, *, 4*, 5x4, or 5x*, as specified by VOTable.\\
1697 delim\hfil\break
1698 \makebox[0pt][l]{\scriptsize\ttfamily xpath:dataType/@delim}&
1699 \footnotesize VARCHAR(*)&
1700 The string that is used to delimit elements of an array value when arraysize is not '1'.\\
1701 param\_use\hfil\break
1702 \makebox[0pt][l]{\scriptsize\ttfamily xpath:@use}&
1703 \footnotesize VARCHAR(*)&
1704 An indication of whether this parameter is required to be provided for the application or service to work properly (one of required, optional, ignored, or NULL).\\
1705 param\_description\hfil\break
1706 \makebox[0pt][l]{\scriptsize\ttfamily xpath:description}&
1707 \footnotesize VARCHAR(*)&
1708 A free-text description of the parameter's contents.\\
1709
1710 \sptablerule
1711 \end{tabular}
1712 \end{inlinetable}
1713
1714
1715 % /GENERATED
1716
1717
1718
1719 The pair \rtent{ivoid}, \rtent{intf\_index} should be an explicit
1720 foreign key into \rtent{interface}.
1721
1722 The remaining requirements and conventions are as per
1723 section \ref{table_table_column}
1724 where applicable, and \rtent{param\_description} taking the role
1725 of \rtent{column\_description}.
1726
1727
1728 % HTML section ends
1729
1730 % subsection table\_intf\_param
1731 % HTML section start
1732
1733 \subsection{The relationship Table}
1734
1735 \label{table_relationship}
1736
1737 The relationship element is a slight denormalization of the
1738 \vorent{vr:Relation\-ship} type: Whereas in VOResource, a single
1739 relationship element can take several IVORNs, in the relational model,
1740 the pairs are stored directly. It is straightforward to translate
1741 between the two representations in the database ingestor.
1742
1743
1744 % GENERATED: maketable.sh rr.relationship
1745
1746 \begin{inlinetable}
1747 \small
1748 \begin{tabular}{p{0.28\textwidth}p{0.2\textwidth}p{0.66\textwidth}}
1749 \sptablerule
1750 \multicolumn{3}{l}{\textit{Column names, utypes, ADQL types, and descriptions for the \rtent{rr.relationship} table}}\\
1751 \sptablerule
1752 ivoid\hfil\break
1753 \makebox[0pt][l]{\scriptsize\ttfamily xpath:/identifier}&
1754 \footnotesize VARCHAR(*)&
1755 The parent resource.\\
1756 relationship\_type\hfil\break
1757 \makebox[0pt][l]{\scriptsize\ttfamily xpath:relationshipType}&
1758 \footnotesize VARCHAR(*)&
1759 The named type of relationship; this can be mirror-of, service-for, served-by, derived-from, related-to.\\
1760 related\_id\hfil\break
1761 \makebox[0pt][l]{\scriptsize\ttfamily xpath:relatedResource/@ivo-id}&
1762 \footnotesize VARCHAR(*)&
1763 The IVOA identifier for the resource referred to.\\
1764 related\_name\hfil\break
1765 \makebox[0pt][l]{\scriptsize\ttfamily xpath:relatedResource}&
1766 \footnotesize VARCHAR(*)&
1767 The name of resource that this resource is related to.\\
1768
1769 \sptablerule
1770 \end{tabular}
1771 \end{inlinetable}
1772
1773
1774 % /GENERATED
1775
1776
1777
1778 The \rtent{ivoid} column should be an explicit foreign key into the
1779 \rtent{resource} table. You should index at least the
1780 \rtent{related\_id} column.
1781
1782 The following columns MUST be lowercased during ingestion:
1783 \rtent{ivoid}, \rtent{relationship\_type},
1784 \rtent{related\_id}.
1785
1786
1787 % HTML section ends
1788
1789 % subsection table\_relationship
1790 % HTML section start
1791
1792 \subsection{The validation Table}
1793
1794 \label{table_validation}
1795
1796 The \rtent{validation} table subsumes the
1797 \vorent{vr:validationLevel}-typed members of both \vorent{vr:Resource}
1798 and \vorent{vr:Capability}.
1799
1800 If the \rtent{cap\_index} column is \texttt{NULL}, the
1801 validation comprises the entire resource. Otherwise, only the
1802 referenced capability has been validated.
1803
1804 While it is recommended that harvesters only accept resource records
1805 from their originating registries, it is valuable to gather validation
1806 results from various sources. Hence, harvesters for the relational
1807 registry may choose to obtain validation data from the OAI-PMH endpoints
1808 of various registries by not harvesting just for the ivo\_managed set and
1809 generate \rtent{rr.validation} rows from these records. This can
1810 trigger potentially problematic behavior when the original registry
1811 updates its resource record in that naive implementations will lose all
1812 third-party validation rows; this may actually be the correct behavior,
1813 since an update of the registry record might very well indicate
1814 validation-relevant changes in the underlying services. Implementations
1815 are free to handle or ignore validation results as they see fit, and
1816 they may add validation results of their own.
1817
1818 The validation levels are defined in \citet{std:RM} and
1819 currently range from 0 (description stored in a registry) to
1820 4 (inspected by a human to be technically and scientifically
1821 correct).
1822
1823
1824 % GENERATED: maketable.sh rr.validation
1825
1826 \begin{inlinetable}
1827 \small
1828 \begin{tabular}{p{0.28\textwidth}p{0.2\textwidth}p{0.66\textwidth}}
1829 \sptablerule
1830 \multicolumn{3}{l}{\textit{Column names, utypes, ADQL types, and descriptions for the \rtent{rr.validation} table}}\\
1831 \sptablerule
1832 ivoid\hfil\break
1833 \makebox[0pt][l]{\scriptsize\ttfamily xpath:/identifier}&
1834 \footnotesize VARCHAR(*)&
1835 The parent resource.\\
1836 validated\_by\hfil\break
1837 \makebox[0pt][l]{\scriptsize\ttfamily xpath:validationLevel/@validatedBy}&
1838 \footnotesize VARCHAR(*)&
1839 The IVOA ID of the registry or organisation that assigned the validation level.\\
1840 val\_level\hfil\break
1841 \makebox[0pt][l]{\scriptsize\ttfamily xpath:validationLevel}&
1842 \footnotesize SMALLINT(1)&
1843 A numeric grade describing the quality of the resource description, when applicable, to be used to indicate the confidence an end-user can put in the resource as part of a VO application or research study.\\
1844 cap\_index\hfil\break
1845 \makebox[0pt][l]{\scriptsize\ttfamily }&
1846 \footnotesize SMALLINT(1)&
1847 If non-NULL, the validation only refers to the capability referenced here.\\
1848
1849 \sptablerule
1850 \end{tabular}
1851 \end{inlinetable}
1852
1853
1854 % /GENERATED
1855
1856
1857
1858 The \rtent{ivoid} column should be an explicit foreign key into
1859 \rtent{resource}. Note, however, that \rtent{ivoid},
1860 \rtent{cap\_index} is \emph{not} a foreign key into \rtent{capability}
1861 since \rtent{cap\_index} may be \texttt{NULL} (in case the validation
1862 addresses the entire resource).
1863
1864 The following columns MUST be lowercased during ingestion:
1865 \rtent{ivoid}, \rtent{validated\_by}.
1866
1867
1868 % HTML section ends
1869
1870 % subsection table\_validation
1871 % HTML section start
1872
1873 \subsection{The res\_date Table}
1874
1875 \label{table_res_date}
1876
1877 The \rtent{res\_date} table contains information gathered from
1878 \vorent{vr:Curation}'s date children.
1879
1880
1881 % GENERATED: maketable.sh rr.res_date
1882
1883 \begin{inlinetable}
1884 \small
1885 \begin{tabular}{p{0.28\textwidth}p{0.2\textwidth}p{0.66\textwidth}}
1886 \sptablerule
1887 \multicolumn{3}{l}{\textit{Column names, utypes, ADQL types, and descriptions for the \rtent{rr.res\_date} table}}\\
1888 \sptablerule
1889 ivoid\hfil\break
1890 \makebox[0pt][l]{\scriptsize\ttfamily xpath:/identifier}&
1891 \footnotesize VARCHAR(*)&
1892 The parent resource.\\
1893 date\_value\hfil\break
1894 \makebox[0pt][l]{\scriptsize\ttfamily xpath:date}&
1895 \footnotesize TIMESTAMP(1)&
1896 A date associated with an event in the life cycle of the resource.\\
1897 value\_role\hfil\break
1898 \makebox[0pt][l]{\scriptsize\ttfamily xpath:date/@role}&
1899 \footnotesize VARCHAR(*)&
1900 A string indicating what the date refers to, e.g., created, availability, updated.\\
1901
1902 \sptablerule
1903 \end{tabular}
1904 \end{inlinetable}
1905
1906
1907 % /GENERATED
1908
1909
1910
1911 The \rtent{ivoid} column should be an explicit foreign key into
1912 \rtent{resource}.
1913
1914 The following columns MUST be lowercased during ingestion:
1915 \rtent{ivoid}, \rtent{value\_role}.
1916
1917
1918 % HTML section ends
1919
1920 % subsection table\_res\_date
1921 % HTML section start
1922
1923 \subsection{The res\_detail Table}
1924
1925 \label{table_res_detail}
1926
1927 The \rtent{res\_detail} table is the relational registry's primary means for
1928 extensibility as well as a fallback for less-used simple
1929 metadata. Conceptually, it stores triples of resource entity
1930 references, resource xpaths,
1931 and values, where resource entities can be resource records themselves
1932 or capabilities. Thus, metadata with values that are either
1933 string-valued or sets of strings can be represented in this table.
1934
1935 As long as the metadata that needs to be represented in the
1936 relational registry for new VOResource extensions is simple enough, no changes to the schema defined
1937 here will be necessary as these are introduced. Instead, the extension itself simply defines
1938 new xpaths to be added in \rtent{res\_detail}.
1939
1940 Some complex metadata -- \vorent{tr:languageFeature} or
1941 \vorent{vstd:key} being examples -- cannot be kept in this table.
1942 If a representation of such information in the relational registry is
1943 required, this standard will need to be changed.
1944
1945 Appendix \ref{d_u_list} gives a list
1946 of resource xpaths from the registry extensions
1947 that were recommendations at the time of writing.
1948 For the resource xpaths marked with an exclamation mark there,
1949 xpath/value pairs MUST be generated whenever the corresponding
1950 metadata items are given in a resource record.
1951 For the remaining resource xpaths, these pairs should be provided if
1952 convenient; they mostly concern test queries and other curation-type
1953 information that, while unlikely to be useful to normal users, may be
1954 relevant to curation-type clients that, e.g., ascertain the continued
1955 operation of services.
1956
1957 Some detail values must be interpreted case-insensitively; this
1958 concerns, in particular, IVORNs like the TAP data model type. For other
1959 rows -- the test queries are immediate examples -- , changing the case
1960 will likely break the data. In order to avoid having to give and
1961 implement case normalization rules by detail xpath, no case normalization
1962 is done on detail values at all, and users and clients will have to use
1963 the \rtent{ivo\_nocasematch} user defined function (see section
1964 \ref{adqludf}) when locating
1965 case-insensitive values. For the resource xpaths given in Appendix \ref{d_u_list}, this concerns all items with xpaths ending
1966 in \texttt{@ivo-id}.
1967
1968 Individual
1969 ingestors
1970 MAY choose to expose additional metadata using other utypes, provided
1971 they are formed according to the rules in
1972 section \ref{vorutypes} (this rule is intended
1973 to minimize the risk of later clashes).
1974
1975 In addition to the metadata listed in this specification,
1976 metadata defined in future
1977 IVOA-approved VOResource extensions MUST or SHOULD be present in
1978 \rtent{res\_details} as the extensions require it.
1979
1980
1981 % GENERATED: maketable.sh rr.res_detail
1982
1983 \begin{inlinetable}
1984 \small
1985 \begin{tabular}{p{0.28\textwidth}p{0.2\textwidth}p{0.66\textwidth}}
1986 \sptablerule
1987 \multicolumn{3}{l}{\textit{Column names, utypes, ADQL types, and descriptions for the \rtent{rr.res\_detail} table}}\\
1988 \sptablerule
1989 ivoid\hfil\break
1990 \makebox[0pt][l]{\scriptsize\ttfamily xpath:/identifier}&
1991 \footnotesize VARCHAR(*)&
1992 The parent resource.\\
1993 cap\_index\hfil\break
1994 \makebox[0pt][l]{\scriptsize\ttfamily }&
1995 \footnotesize SMALLINT(1)&
1996 The index of the parent capability; if NULL the xpath-value pair describes a member of the entire resource.\\
1997 detail\_xpath\hfil\break
1998 \makebox[0pt][l]{\scriptsize\ttfamily }&
1999 \footnotesize VARCHAR(*)&
2000 The xpath of the data item.\\
2001 detail\_value\hfil\break
2002 \makebox[0pt][l]{\scriptsize\ttfamily }&
2003 \footnotesize VARCHAR(*)&
2004 (Atomic) value of the member.\\
2005
2006 \sptablerule
2007 \end{tabular}
2008 \end{inlinetable}
2009
2010
2011 % /GENERATED
2012
2013
2014
2015 The \rtent{ivoid} column should be an explicit foreign key into
2016 \rtent{resource}. It is recommended to maintain indexes on
2017 at least the columns
2018 \rtent{detail\_xpath} and \rtent{detail\_value}, where the
2019 index on \rtent{detail\_value} should ideally work for both direct
2020 comparisons and searches using the \rtent{ivo\_nocasematch}
2021 function.
2022
2023 The following column MUST be lowercased during ingestion:
2024 \rtent{ivoid}. Clients are advised to
2025 use the \rtent{ivo\_nocasematch} function to search in
2026 \rtent{detail\_value} if the values are to be compared
2027 case-insensitively (e.g., all IVORNs).
2028
2029
2030 % HTML section ends
2031
2032 % subsection table\_res\_detail
2033 % HTML section ends
2034
2035 % section vortables
2036 % HTML section start
2037
2038 \section{ADQL User Defined Functions}
2039
2040 \label{adqludf}
2041
2042 TAP Servers implementing the
2043 \texttt{ivo://ivoa.net/std/RegTAP\#1.0} data model MUST
2044 implement the following four functions in their ADQL 2.0 language,
2045 with signatures written as recommended in \citet{std:TAPREGEXT}:
2046
2047
2048 \begin{bigdescription}
2049 \term[\small\texttt{ivo\_nocasematch(value VARCHAR(*), pat VARCHAR(*))->INTEGER}]
2050 The function returns 1 if \texttt{pat} matches
2051 \texttt{value}, 0 otherwise.
2052 \texttt{pat} is defined as for the SQL LIKE operator, but the
2053 match is performed case-insensitively.
2054
2055 \term[\small\texttt{ivo\_hasword(haystack VARCHAR(*), needle VARCHAR(*)) -> INTEGER}]The function takes two strings and returns 1 if the second is
2056 contained in the first one in a ``word'' sense, i.e., delimited by
2057 non-letter characters or the beginning or end of the string, where case
2058 is ignored.
2059 Additionally, servers MAY employ techniques to improve recall, in
2060 particular stemming. Registry clients must hence expect different results
2061 from different servers when using \rtent{ivo\_hasword}; for such
2062 queries trying them on multiple registries may improve recall.
2063 \term[\small\texttt{ivo\_hashlist\_has(hashlist VARCHAR(*), item VARCHAR(*)) -> INTEGER}]The function takes two strings; the first is a list of words not
2064 containing the hash sign (\#), concatenated by hash signs, the second is
2065 a word not containing the hash sign. It returns 1 if, compared
2066 case-insensitively, the second argument is in the list of words encoded in
2067 the first argument. The behavior for second
2068 arguments containing a hash sign is undefined.
2069 \term[\small\texttt{ivo\_string\_agg(expr VARCHAR(*), deli VARCHAR(*)) -> VARCHAR(*)}]An aggregate function returning all values of
2070 \texttt{expr} within a GROUP concatenated with
2071 \texttt{delim}. NULLs in the aggregate do not contribute, an empty
2072 aggregate yields an empty string.
2073
2074 \end{bigdescription}
2075
2076 Reference implementations of the four functions for the PostgreSQL
2077 database system are given in Appendix \ref{appPGDefs}.
2078
2079
2080 % HTML section ends
2081
2082 % section adqludf
2083 % HTML section start
2084
2085 \section{Common Queries to the Relational Registry}
2086
2087 \label{sample_queries}
2088
2089 This section contains sample queries to the relational registry,
2090 mostly contributed as use cases by various members of the IVOA Registry
2091 working group. They are intended as an aid in designing relational
2092 registry queries, in particular for users new to the data model.
2093
2094 When locating access URLs for capabilities of standard services, these
2095 sample queries look for interfaces of type \vorent{vs:ParamHTTP} within
2096 the embedding capabilities. This is not strictly as intended by
2097 VOResource, which has the special \vorent{role} attribute to mark the
2098 interface on which a standard protocol is exposed within a capability.
2099 In RegTAP, this method of locating standard interfaces would be effected
2100 by constraining \texttt{intf\_role LIKE 'std\%'}. In actual VO
2101 practice, too many standard interfaces are lacking a proper declaration
2102 of their role, and actual clients locate the standard interfaces as
2103 given here. Following widespread practice client designers are
2104 encouraged to compare against the interface types rather than rely on
2105 \vorent{interface/@role}, and resource record authors should make sure
2106 clients can discover standard interfaces both by the interfaces' roles
2107 and types.
2108
2109
2110 \subsection{TAP accessURLs}
2111 \textbf{Problem:} Find all TAP services; return their accessURLs
2112
2113 As the capability type is in
2114 \rtent{rr.capability}, whereas the access URL can be
2115 found from
2116 \rtent{rr.interface}, this requires
2117 a join:
2118
2119 %CHECK_HERE
2120 \begin{lstlisting}[language=SQL,flexiblecolumns=true]
2121 SELECT ivoid, access_url
2122 FROM rr.capability
2123 NATURAL JOIN rr.interface
2124 WHERE standard_id like 'ivo://ivoa.net/std/tap%'
2125 AND intf_type='vs:paramhttp'
2126 \end{lstlisting}
2127
2128 Other \rtent{standard\_id}s relevant here include:
2129
2130
2131 \begin{itemize}
2132
2133 \item \texttt{ivo://ivoa.net/std/registry} for OAI-PMH services,{}
2134
2135 \item \texttt{ivo://ivoa.net/std/sia} for SIA services,{}
2136
2137 \item \texttt{ivo://ivoa.net/std/conesearch} for SCS services,
2138 and{}
2139
2140 \item \texttt{ivo://ivoa.net/std/ssa} for SSA services.{}
2141
2142 \end{itemize}
2143
2144 \subsection{Image Services with Spirals}
2145
2146 \textbf{Problem:} Find all SIA services that might have spiral
2147 galaxies
2148
2149 This is somewhat tricky since it is probably hard to image a part
2150 of the sky guaranteed not to have some, possibly distant, spiral galaxy
2151 in it. However, translating the intention into ``find all SIA services
2152 that mention spiral in either the subject (from
2153 \rtent{rr.res\_subject}), the description, or the
2154 title (which are in
2155 \rtent{rr.resource})'',
2156 the query would become:
2157
2158
2159 %CHECK_HERE
2160 \begin{lstlisting}[language=SQL,flexiblecolumns=true]
2161 SELECT ivoid, access_url
2162 FROM rr.capability
2163 NATURAL JOIN rr.resource
2164 NATURAL JOIN rr.interface
2165 NATURAL JOIN rr.res_subject
2166 WHERE standard_id='ivo://ivoa.net/std/sia'
2167 AND intf_type='vs:paramhttp'
2168 AND (
2169 1=ivo_nocasematch(res_subject, '%spiral%')
2170 OR 1=ivo_hasword(res_description, 'spiral')
2171 OR 1=ivo_hasword(res_title, 'spiral'))
2172 \end{lstlisting}
2173
2174
2175 \subsection{Infrared Image Services}
2176
2177 \textbf{Problem:} Find all SIA services that provide infrared
2178 images
2179
2180 The waveband information in
2181 \rtent{rr.resource}
2182 comes in hash-separated atoms (which can be
2183 radio, millimeter, infrared, optical, uv, euv, x-ray, or gamma-ray).
2184 For matching those, use the \rtent{ivo\_hashlist\_has} function as
2185 below. The access URL and the service standard come from
2186 \rtent{rr.interface} and
2187 \rtent{rr.capability}, respectively.
2188
2189
2190 %CHECK_HERE
2191 \begin{lstlisting}[language=SQL,flexiblecolumns=true]
2192 SELECT ivoid, access_url
2193 FROM rr.capability
2194 NATURAL JOIN rr.resource
2195 NATURAL JOIN rr.interface
2196 WHERE standard_id='ivo://ivoa.net/std/sia'
2197 AND intf_type='vs:paramhttp'
2198 AND 1=ivo_hashlist_has('infrared', waveband)
2199 \end{lstlisting}
2200
2201 \subsection{Catalogs with Redshifts}
2202 \textbf{Problem:} Find all searchable catalogs (i.e., cone search
2203 services) that provide a column containing redshifts
2204
2205 Metadata on columns exposed by a service are contained in
2206 \rtent{rr.table\_column}. Again, this table can be
2207 naturally joined with
2208 \rtent{rr.capability} and
2209 \rtent{rr.interface}.
2210 :
2211 %CHECK_HERE
2212 \begin{lstlisting}[language=SQL,flexiblecolumns=true]
2213 SELECT ivoid, access_url
2214 FROM rr.capability
2215 NATURAL JOIN rr.table_column
2216 NATURAL JOIN rr.interface
2217 WHERE standard_id='ivo://ivoa.net/std/conesearch'
2218 AND intf_type='vs:paramhttp'
2219 AND ucd='src.redshift'
2220 \end{lstlisting}
2221
2222 Sometimes you want to match a whole set of ucds. Frequently the
2223 simple regular expressions of SQL will help, as in
2224 \texttt{AND ucd LIKE 'pos.parallax\%'}. When that is not enough,
2225 use boolean OR expressions
2226
2227 \subsection{Names from an Authority}
2228
2229 \textbf{Problem:} Find all the resources published by a certain
2230 authority
2231
2232 An ``authority'' within the VO is something that hands out identifiers.
2233 You can tell what authority a record came from by looking at the ``host
2234 part'' of the IVO identifier, most naturally obtained from
2235 \rtent{rr.resource}. Since ADQL cannot actually parse
2236 URIs, we make do with simple string matching:
2237
2238
2239 %CHECK_HERE
2240 \begin{lstlisting}[language=SQL,flexiblecolumns=true]
2241 SELECT ivoid
2242 FROM rr.resource
2243 WHERE ivoid LIKE 'ivo://org.gavo.dc%'
2244 \end{lstlisting}
2245
2246 \subsection{Records Published by X}
2247
2248 \textbf{Problem:} What registry records are there from a given
2249 publisher?
2250
2251 This uses the
2252 \rtent{rr.res\_role}
2253 table both to
2254 match names (in this case, a publisher that has ``gavo'' in its name) and
2255 to ascertain the named entity actually publishes the resource (rather
2256 than, e.g., just being the contact in case of trouble). The result is a
2257 list of ivoids in this case. You could join this with any other
2258 table in the relational registry to find out more about these
2259 services.
2260
2261
2262 %CHECK_HERE
2263 \begin{lstlisting}[language=SQL,flexiblecolumns=true]
2264 SELECT ivoid
2265 FROM rr.res_role
2266 WHERE 1=ivo_nocasematch(role_name, '%gavo%')
2267 AND base_role='publisher'
2268 \end{lstlisting}
2269
2270 or, if the publisher actually gives its ivo-id in the registry
2271 records,
2272
2273
2274 %CHECK_HERE
2275 \begin{lstlisting}[language=SQL,flexiblecolumns=true]
2276 SELECT ivoid
2277 FROM rr.res_role
2278 WHERE role_ivoid='ivo://ned.ipac/ned'
2279 AND base_role='publisher'
2280 \end{lstlisting}
2281
2282 \subsection{Records from Registry}
2283
2284 \textbf{Problem:} What registry records are
2285 there originating from registry X?
2286
2287 This is mainly a query interesting for registry maintainers. Still,
2288 it is a nice example for joining with the
2289 \rtent{rr.res\_detail} table, in this case to
2290 first get a list of all authorities managed by the CDS registry.
2291
2292
2293 %CHECK_HERE
2294 \begin{lstlisting}[language=SQL,flexiblecolumns=true]
2295 SELECT ivoid FROM rr.resource
2296 RIGHT OUTER JOIN (
2297 SELECT 'ivo://' || detail_value || '%' AS pat
2298 FROM rr.res_detail
2299 WHERE detail_xpath='/managedAuthority'
2300 AND ivoid='ivo://cds.vizier/registry')
2301 AS authpatterns
2302 ON (1=ivo_nocasematch(resource.ivoid, authpatterns.pat))
2303 \end{lstlisting}
2304
2305 \subsection{Locate RegTAP services}
2306
2307 \textbf{Problem:} Find all TAP endpoints offering the relational
2308 registry
2309
2310 This is the discovery query for RegTAP services themselves; note how
2311 this combines
2312 \rtent{rr.res\_detail} pairs with
2313 \rtent{rr.capability}
2314 and
2315 \rtent{rr.interface} to locate the desired protocol
2316 endpoints. As clients should not usally be concerned with minor
2317 versions of protocols unless they rely on additions made in later
2318 versions, this query will return endpoints supporting ``version 1'' rather
2319 than exactly version 1.0.
2320
2321
2322 %CHECK_HERE
2323 \begin{lstlisting}[language=SQL,flexiblecolumns=true]
2324 SELECT access_url
2325 FROM rr.interface
2326 NATURAL JOIN rr.capability
2327 NATURAL JOIN rr.res_detail
2328 WHERE standard_id='ivo://ivoa.net/std/tap'
2329 AND intf_type='vs:paramhttp'
2330 AND detail_xpath='/capability/dataModel/@ivo-id'
2331 AND 1=ivo_nocasematch(detail_value,
2332 'ivo://ivoa.net/std/regtap#1.%')
2333 \end{lstlisting}
2334
2335 \subsection{TAP with Physics}
2336
2337 \textbf{Problem:} Find all TAP services
2338 exposing a table with certain features
2339
2340 ``Certain features'' could be ``have some word in their description
2341 and having a column with a certain UCD''. Either way, this kind of query
2342 fairly invariably combines the usual
2343 \rtent{rr.capability} and
2344 \rtent{rr.interface}
2345 for service location with
2346 \rtent{rr.table\_column}
2347 for the column metadata
2348 and
2349 \rtent{rr.res\_table} for properties of tables.
2350
2351
2352 %CHECK_HERE
2353 \begin{lstlisting}[language=SQL,flexiblecolumns=true]
2354 SELECT ivoid, access_url, name, ucd, column_description
2355 FROM rr.capability
2356 NATURAL JOIN rr.interface
2357 NATURAL JOIN rr.table_column
2358 NATURAL JOIN rr.res_table
2359 WHERE standard_id='ivo://ivoa.net/std/tap'
2360 AND intf_type='vs:paramhttp'
2361 AND 1=ivo_hasword(table_description, 'quasar')
2362 AND ucd='phot.mag;em.opt.v'
2363 \end{lstlisting}
2364
2365 \subsection{Theoretical SSA}
2366
2367 \textbf{Problem:} Find all SSAP services that
2368 provide theoretical spectra.
2369
2370 The metadata required to solve this problem is found in the SSAP
2371 registry extension and is thus kept in
2372 \rtent{rr.res\_detail}:
2373
2374
2375 %CHECK_HERE
2376 \begin{lstlisting}[language=SQL,flexiblecolumns=true]
2377 SELECT access_url
2378 FROM rr.res_detail
2379 NATURAL JOIN rr.capability
2380 NATURAL JOIN rr.interface
2381 WHERE detail_xpath='/capability/dataSource'
2382 AND intf_type='vs:paramhttp'
2383 AND standard_id='ivo://ivoa.net/std/ssa'
2384 AND detail_value='theory'
2385 \end{lstlisting}
2386
2387
2388 \subsection{Find Contact Persons}
2389
2390 \textbf{Problem:} The service at
2391 \texttt{http://dc.zah.uni-heidelberg.de/tap} is down, who can
2392 fix it?
2393
2394 This uses the \rtent{rr.res\_role} table and returns all information on
2395 it based on the IVORN of a service that in turn was obtained from
2396 \rtent{rr.interface}. You could restrict to the actual technical
2397 contact person by requiring \texttt{base\_role='contact'}.
2398
2399
2400 %CHECK_HERE
2401 \begin{lstlisting}[language=SQL,flexiblecolumns=true]
2402 SELECT DISTINCT base_role, role_name, email
2403 FROM rr.res_role
2404 NATURAL JOIN rr.interface
2405 WHERE access_url='http://dc.zah.uni-heidelberg.de/tap'
2406 \end{lstlisting}
2407
2408 \subsection{Related Capabilities}
2409
2410 \textbf{Problem:} Get the capabilities of all services serving a
2411 specific resource (typically, a data collection).
2412
2413 In the VO, data providers can register pure data collections without
2414 access options (or just furnished with a link to a download). They can
2415 then declare that their data can be ``served-by'' some other resource,
2416 typically a TAP service or some collective service for a number of
2417 instruments. To locate the access options to the data itself, inspect
2418 \rtent{rr.relationship} and use it to select records
2419 from
2420 \rtent{rr.capability}.
2421
2422 %CHECK_HERE
2423 \begin{lstlisting}[language=SQL,flexiblecolumns=true]
2424 SELECT *
2425 FROM rr.relationship AS a
2426 JOIN rr.capability AS b
2427 ON (a.related_id=b.ivoid)
2428 WHERE relationship_type='served-by'
2429 \end{lstlisting}
2430
2431
2432 % HTML section ends
2433
2434 % section sample\_queries
2435
2436 \appendix
2437
2438 \section{XPaths for res\_details}
2439
2440 \label{d_u_list}
2441
2442 This appendix defines the \rtent{res\_details}
2443 table (see section \ref{table_res_detail} for
2444 details) by giving
2445 xpaths for which xpath/value pairs MUST (where marked with an
2446 exclamation mark) or SHOULD be given if the
2447 corresponding data is present in the resource records. This list is
2448 normative for metadata defined in IVOA recommendations current as of the
2449 publication of this document (see section \ref{rolewithinivoa}).
2450 As laid down in section \ref{table_res_detail},
2451 new VOResource extensions or new
2452 versions of existing VOResource extensions may amend this list.
2453
2454 In case there are conflicts between this list and xpaths derived
2455 from schema files using the rules given in section \ref{vorutypes}, the conflict must be considered due to an
2456 editorial oversight in the preparation of this list, and the xpaths from the
2457 schema files are normative. Errata to this list will be issued in such
2458 cases.
2459
2460 The xpaths are sufficient for locating the respective metadata as per
2461 section \ref{vorutypes}. With the explanations we
2462 give the canonical prefixes for the XML namespaces from which the items
2463 originate, which is where further information can be found.
2464
2465
2466 \begin{description}
2467 \item[/accessURL (!)]For data collection resources, this is
2468 the URL that can be used to download the data contained. Do
2469 \emph{not} enter accessURLs from interfaces into res\_detail (vs).
2470 \item[/capability/executionDuration/hard]The hard run time limit, given in seconds (tr).
2471 \item[/capability/complianceLevel]The category indicating the level to which this instance complies with the SSA standard (ssap).
2472 \item[/capability/creationType (!)]The category that describes the process used to produce the dataset; one of archival, cutout, filtered, mosaic, projection, specialExtraction, catalogExtraction (ssap).
2473 \item[/capability/dataModel (!)]The short, human-readable name of a data model supported by a TAP service; for most applications, clients should rather constrain /capability/dataModel/@ivo-id (tr).
2474 \item[/capability/dataModel/@ivo-id (!)]The IVORN of the data model supported by a TAP service (tr).
2475 \item[/capability/dataSource (!)]The category specifying where the data originally came from; one of survey, pointed, custom, theory, artificial (ssap).
2476 \item[/capability/defaultMaxRecords (!)]The largest number of records that the service will return when the MAXREC parameter is not specified in the query input (ssap).
2477 \item[/capability/executionDuration/default]The run time limit for newly-created jobs, given in seconds (tr).
2478 \item[/capability/imageServiceType (!)]The class of image service: Cutout, Mosaic, Atlas, Pointed (sia).
2479 \item[/capability/interface/securityMethod/@standardID (!)]A description of a security mechanism. Although this really is a property of an interface, the reference here goes to the embedding capability, which is sufficient for discovery of the existence of such interfaces. If the capability holds multiple interfaces, clients need other means (e.g., VOSI capabilities) to figure out the mapping between access URLs and supported security methods.
2480 \item[/capability/language/name (!)]A short, human-readable name of a language understood by the TAP service (tr).
2481 \item[/capability/language/version/@ivo-id (!)]The IVORN of a language supported by a TAP service (tr).
2482 \item[/capability/maxAperture]The largest aperture that can be supported upon request via the APERTURE input parameter by a service that supports the special extraction creation method (ssap).
2483 \item[/capability/maxFileSize (!)]The maximum image file size in bytes (sia).
2484 \item[/capability/maxImageExtent/lat]The maximum size in the latitude (Dec.) direction (sia).
2485 \item[/capability/maxImageExtent/long]The maximum size in the longitude (R.A.) direction (sia).
2486 \item[/capability/maxImageSize/lat]The image size in the latitude (Dec.) direction in pixels (sia-1.0).
2487 \item[/capability/maxImageSize/long]The image size in the longitude (R.A.) direction in pixels (sia-1.0).
2488 \item[/capability/maxImageSize]A measure of the largest image the service can produce given as the maximum number of pixels along the first or second axes. (sia).
2489 \item[/capability/maxQueryRegionSize/lat]The maximum size in the latitude (Dec.) direction (sia).
2490 \item[/capability/maxQueryRegionSize/long]The maximum size in the longitude (R.A.) direction (sia).
2491 \item[/capability/maxRecords (!)]The largest number of items (records, rows, etc.) that the service will return (cs, sia, vg, ssap).
2492 \item[/capability/maxSearchRadius (!)]The largest search radius, in degrees, that will be accepted by the service without returning an error condition. Not providing this element or specifying a value of 180 indicates that there is no restriction. (ssap)
2493 \item[/capability/maxSR (!)]The largest search radius of a cone search service (cs).
2494 \item[/capability/outputFormat/@ivo-id (!)]An IVORN of an output format (tr).
2495 \item[/capability/outputFormat/alias]A short, mnemonic identifier for a service's output format (tr).
2496 \item[/capability/outputFormat/mime (!)]The MIME type of an output format (tr).
2497 \item[/capability/outputLimit/default]The maximal output size for newly-created jobs (tr).
2498 \item[/capability/outputLimit/default/@unit]The unit (rows/bytes) in which the service's default output limit is given (tr).
2499 \item[/capability/outputLimit/hard]The hard limit of a service's output size (tr).
2500 \item[/capability/outputLimit/hard/@unit]The unit of this service's hard output limit (tr).
2501 \item[/capability/retentionPeriod/default]The default time between job creation and removal on this service, given in seconds (tr).
2502 \item[/capability/retentionPeriod/hard]The hard limit for the retention time of jobs on this services (tr).
2503 \item[/capability/supportedFrame (!)]The STC name for a world coordinate system frame supported by this service (ssap).
2504 \item[/capability/testQuery/catalog]The catalog to query (cs).
2505 \item[/capability/testQuery/dec]Declination in a test query (cs)
2506 \item[/capability/testQuery/extras]Any extra (non-standard) parameters that must be provided (apart from what is part of base URL given by the accessURL element; cs, sia).
2507 \item[/capability/testQuery/pos/lat]The Declination of the center of the search position in decimal degrees (ssap, sia).
2508 \item[/capability/testQuery/pos/long]The Right Ascension of the center of the search position in decimal degrees (ssap, sia).
2509 \item[/capability/testQuery/pos/refframe]A coordinate system reference frame name for a test query. If not provided, ICRS is assumed (ssap).
2510 \item[/capability/testQuery/queryDataCmd]Fully specified test query formatted as an URL argument list in the syntax specified by the SSA standard. The list must exclude the REQUEST argument (ssap).
2511 \item[/capability/testQuery/ra]Right ascension in a test query (cs).
2512 \item[/capability/testQuery/size]The size of the search radius in an SSA search query (ssap).
2513 \item[/capability/testQuery/size/lat]Region size for a SIA test query in declination (sia).
2514 \item[/capability/testQuery/size/long]Region size for a SIA test query in RA (sia).
2515 \item[/capability/testQuery/sr]Search radius of a cone search service's test query (cs).
2516 \item[/capability/testQuery/verb]Verbosity of a service's test query (cs, sia).
2517 \item[/capability/uploadLimit/default]An advisory size above which user agents should reconfirm uploads to this service (tr).
2518 \item[/capability/uploadLimit/default/@unit]The unit of the limit specified (tr).
2519 \item[/capability/uploadLimit/hard]Hard limit for the size of uploads on this service (tr).
2520 \item[/capability/uploadLimit/hard/@unit]The unit of the limit specified (tr).
2521 \item[/capability/uploadMethod/@ivo-id]The IVORN of an upload method supported by the service (tr).
2522 \item[/capability/verbosity (!)]\texttt{true} if the service supports the VERB keyword; \texttt{false}, otherwise (cs).
2523 \item[/coverage/footprint (!)]A URL of a footprint service for retrieving precise and up-to-date description of coverage (vs).
2524 \item[/coverage/footprint/@ivo-id (!)]The URI form of the IVOA identifier for the service describing the capability refered to by this element (vs).
2525 \item[/deprecated (!)]A sentinel that all versions of the referenced standard are deprecated. The value is a human-readable explanation for the designation (vstd).
2526 \item[/endorsedVersion (!)]A version of a standard that is recommended for use (vstd).
2527 \item[/facility (!)]The observatory or facility used to collect the data contained or managed by this resource (vs).
2528 \item[/format (!)]The physical or digital manifestation of the information supported by a (DataCollection) resource. MIME types should be used for network-retrievable, digital data, non-MIME type values are used for media that cannot be retrieved over the network (vs).
2529 \item[/format/@isMIMEType]If \texttt{true}, then an accompanying \texttt{/format} item is a MIME Type. Within \rtent{res\_details}, this does not work for services that give more than one format; since furthermore the literal of \vorent{vs:Format} allows a good guess whether or not it is a MIME type, this does not appear a dramatic limitation (vs).
2530 \item[/full]If true, the registry attempts to collect all resource records known to the IVOA (vg).
2531 \item[/instrument (!)]The instrument used to collect the data contained or managed by a resource (vr).
2532 \item[/instrument/@ivo-id (!)]IVORN of the instrument used to collect the data contained or managed by a resource (vr).
2533 \item[/managedAuthority (!)]An authority identifier managed by a registry (vg).
2534 \item[/managingOrg (!)]The organization that manages or owns this authority (vg).
2535 \item[/schema/@namespace (!)]An identifier for a schema described by a standard (vstd).
2536
2537 \end{description}
2538
2539 Note that the representation of StandardsRegExt's
2540 \vorent{status} and \vorent{use}
2541 attributes as well as its \vorent{key} would require sequences of
2542 complex objects, which is impossible using \rtent{res\_detail}.
2543 Hence, the respective metadata is not queriable
2544 within the relational registry. Similarly, complex TAPRegExt metadata on
2545 languages, user defined functions, and the like cannot be
2546 represented in this table. Since these pieces of metadata do not seem
2547 relevant to resource discovery and are geared towards other uses of the
2548 respective VOResource extensions, a more complex model does not seem
2549 warranted just so they can be exposed.
2550
2551
2552 % HTML section ends
2553
2554 % section d\_u\_list
2555 % HTML section start
2556
2557 \section{The Extra UDFs in
2558 PL/pgSQL}
2559
2560 \label{appPGDefs}
2561
2562 What follows are (non-normative)
2563 implementations of three of the user defined functions
2564 specificed in section \ref{adqludf} in the SQL dialect
2565 of PostgreSQL (e.g., \citet{doc:Postgres92}).
2566
2567 Note that PostgreSQL cannot use fulltext indexes on the respective
2568 columns if the functions are defined in this way for (fairly subtle)
2569 reasons connected with NULL value handling. While workarounds are
2570 conceivable, they come with potentially unwelcome side effects, at least
2571 as of PostgreSQL 9.x. It is therefore recommended to replace
2572 expressions involving the functions given here with simple boolean
2573 expressions in the ADQL translation layer whenever possible.
2574
2575
2576 \begin{lstlisting}[language=SQL,flexiblecolumns=true]
2577 CREATE OR REPLACE FUNCTION
2578 ivo_hasword(haystack TEXT, needle TEXT)
2579 RETURNS INTEGER AS $func$
2580 SELECT CASE WHEN to_tsvector($1) @@ plainto_tsquery($2)
2581 THEN 1
2582 ELSE 0
2583 END
2584 $func$ LANGUAGE SQL;
2585
2586 CREATE OR REPLACE FUNCTION
2587 ivo_hashlist_has(hashlist TEXT, item TEXT)
2588 RETURNS INTEGER AS $func$
2589 -- postgres can't RE-escape a user string; hence, we'll have
2590 -- to work on the hashlist (this assumes hashlist is already
2591 -- lowercased).
2592 SELECT CASE WHEN lower($2) = ANY(string_to_array($1, '#'))
2593 THEN 1
2594 ELSE 0
2595 END
2596 $func$ LANGUAGE SQL;
2597
2598 CREATE OR REPLACE FUNCTION
2599 ivo_nocasematch(value TEXT, pattern TEXT)
2600 RETURNS INTEGER AS $func$
2601 SELECT CASE WHEN $1 ILIKE $2
2602 THEN 1
2603 ELSE 0
2604 END
2605 $func$ LANGUAGE SQL;
2606
2607 -- ivo_string_agg directly corresponds to string_agg; this translation
2608 -- should be done in the ADQL translator.
2609 \end{lstlisting}
2610
2611
2612 % HTML section ends
2613
2614 % section appPGDefs
2615 % HTML section start
2616
2617 \section{Implementation notes}
2618
2619 \label{appBP}
2620
2621 This appendix contains a set of constraints and recommendations for
2622 implementing the relational registry model on actual RDBMSes, originating
2623 partly from an analysis of the data content of the VO registry in February
2624 2013, partly from a consideration of limits in various XML schema documents.
2625 This concerns, in particular, minimum lengths for columns of type
2626 adql:VARCHAR. Implementations MUST NOT truncate strings of length equal
2627 to or smaller than the minimal lengths given here; the limitations are
2628 not, however, upper limits, and indeed, when choosing an implementation
2629 strategy it is in general preferable to not impose artificial length
2630 restrictions, in particular if no performance penalty is incurred.
2631
2632 These notes can also be useful with a view to preparing user interfaces for
2633 the relational registry, since input forms and similar user interface
2634 elements invariably have limited space; the limits here give reasonable
2635 defaults for the amount of data that should minimally be manipulatable
2636 by a user with reasonable effort.
2637
2638 The \rtent{ivoid} field present in every table of this
2639 specification merits special consideration, on the one hand due to its
2640 frequency, but also since other IVOA identifiers stored in the
2641 relational registry should probably be treated analogously.
2642 Given that IVORNs in the 2013 data fields have a maximum
2643 length of roughly 100 characters, we propose that a maximum length of
2644 255 should be sufficient even when taking possible fragment identifiers
2645 into account.
2646
2647 \begin{inlinetable}
2648 \begin{tabular}{llp{6cm}}
2649 \sptablerule
2650 \textbf{Field type}&
2651 \textbf{Datatype}&
2652 \textbf{Pertinent Fields}\\
2653 \sptablerule
2654 ivo-id&\texttt{VARCHAR(255)}&
2655 {all\_tables}.ivoid\hfil\break
2656 res\_role.role\_ivoid\hfil\break
2657 capability.standard\_id\hfil\break
2658 relationship.related\_id\hfil\break
2659 validation.validated\_by\hfil\break
2660 res\_detail.detail\_value for several values of detail\_xpath\\
2661 \sptablerule
2662 \end{tabular}
2663 \end{inlinetable}
2664
2665 The relational registry also contains some date-time values. The most
2666 straightforward implementation certainly is to use SQL timestamps.
2667 Other relational registry fields that straightforwardly map to common
2668 SQL types are those that require numeric values, viz.,
2669 \texttt{REAL}, \texttt{SMALLINT}, and
2670 \texttt{INTEGER}. The following table summarizes these:
2671
2672
2673 \begin{inlinetable}
2674 \begin{tabular}{llp{6cm}}
2675 \sptablerule
2676 \textbf{Field type}&
2677 \textbf{Datatype}&
2678 \textbf{Pertinent Fields}\\
2679 \sptablerule
2680 floating point&\texttt{REAL}&resource.region\_of\_regard\\
2681 \sptablerule
2682 small integer&\texttt{SMALLINT}&table\_column.std\hfil\break
2683 intf\_param.std\hfil\break
2684 validation.level\\
2685 \sptablerule
2686 \end{tabular}
2687 \end{inlinetable}
2688
2689 The fields containing Utypes, UCDs, and Units are treated in parallel
2690 here. The 2013 registry data indicates that a length of 128 characters is
2691 sufficient for real-world purposes; actually, at least UCDs and Units
2692 could of course grow without limitations, but it seems unreasonable
2693 anything longer than a typical line might actually be useful. As far as
2694 utypes are concerned, we expect those to shrink rather than grow with
2695 new standardization efforts.
2696
2697 In this category, we also summarize our resource xpaths. With the
2698 conventions laid down in this document, it seems unlikely that future
2699 extensions to VOResource will be so deeply nested that 128 characters
2700 will not be sufficient for their resource xpaths.
2701
2702 \begin{inlinetable}
2703 \begin{tabular}{llp{6cm}}
2704 \sptablerule
2705 \textbf{Field type}&
2706 \textbf{Datatype}&
2707 \textbf{Pertinent Fields}\\
2708 \sptablerule
2709 Utype&\texttt{VARCHAR(128)}&res\_schema.schema\_utype\hfil\break
2710 res\_table.table\_utype\hfil\break
2711 table\_column.utype\hfil\break
2712 intf\_param.utype\\
2713 \sptablerule
2714 UCD&\texttt{VARCHAR(128)}&
2715 table\_column.ucd\hfil\break
2716 intf\_param.ucd\\
2717 \sptablerule
2718 Unit&\texttt{VARCHAR(128)}&
2719 table\_column.unit\hfil\break
2720 intf\_param.unit\\
2721 \sptablerule
2722 xpath&\texttt{VARCHAR(128)}&
2723 res\_detail.detail\_xpath \\
2724 \sptablerule
2725 \end{tabular}
2726 \end{inlinetable}
2727
2728 The relational registry further has an
2729 e-mail field, for which we chose 128 characters as a reasonable upper
2730 limit (based on a real maximum of 40 characters in the 2013 data).
2731 There are furthermore URLs (in addition to access and reference URLs,
2732 there are also URLs for the WSDL of SOAP services and logos for roles).
2733 Due to the importance of in particular the access URLs we strongly
2734 recommend to use non-truncating types here. Empirically, there are
2735 access URLs of up to 224 characters in 2013 (reference URLs are less
2736 critical at a maximum of 96 characters). Expecting that with REST-based
2737 services, URL lengths will probably rather tend down than up, we still
2738 permit truncation at 255 characters.
2739
2740
2741 \begin{inlinetable}
2742 \begin{tabular}{llp{6cm}}
2743 \sptablerule
2744 \textbf{Field type}&
2745 \textbf{Datatype}&
2746 \textbf{Pertinent Fields}\\
2747 \sptablerule
2748 e-mail&\texttt{VARCHAR(128)}&
2749 res\_role.email\\
2750 \sptablerule
2751 URLs&\texttt{VARCHAR(255)}&resource.reference\_url\hfil\break
2752 res\_role.logo\hfil\break
2753 interface.wsdl\_url\hfil\break
2754 interface.access\_url \\
2755 \sptablerule
2756 \end{tabular}
2757 \end{inlinetable}
2758
2759 The next group of columns comprises those that have values taken from
2760 a controlled or finite vocabulary.
2761 Trying to simplify the view,
2762 lengths in the form
2763 of powers of two are considered.
2764
2765 \begin{inlinetable}
2766 \begin{tabular}{llp{6cm}}
2767 \sptablerule
2768 \textbf{Field type}&
2769 \textbf{Datatype}&
2770 \textbf{Pertinent Fields}\\
2771 \sptablerule
2772 long enumerations&\texttt{VARCHAR(255)}&resource.content\_level\hfil\break
2773 resource.content\_type\\
2774 \sptablerule
2775 short enumerations&\texttt{VARCHAR(64)}&
2776 resource.rights\hfil\break
2777 resource.waveband\\
2778 \sptablerule
2779 type names&\texttt{VARCHAR(32)}&resource.res\_type\hfil\break
2780 capability.cap\_type\hfil\break
2781 res\_table.table\_type\hfil\break
2782 table\_column.flag\hfil\break
2783 table\_column.datatype\hfil\break
2784 table\_column.extended\_schema\hfil\break
2785 table\_column.extended\_type\hfil\break
2786 table\_column.type\_system\hfil\break
2787 interface.result\_type\hfil\break
2788 intf\_param.datatype\hfil\break
2789 intf\_param.extended\_schema\hfil\break
2790 intf\_param.extended\_type\\
2791 \sptablerule
2792 short terms&\texttt{VARCHAR(4)}&interface.query\_type\hfil\break
2793 interface.url\_use\\
2794 \sptablerule
2795 \end{tabular}
2796 \end{inlinetable}
2797
2798
2799 Finally, there are the fields that actually contain what is
2800 basically free text.
2801 For these, we have made a choice from reasonable powers of two lengths
2802 considering the actual lengths in the 2013 registry data.
2803 A special case are fields that either contain natural language text (the
2804 descriptions) or those that have grown without limit historically
2805 (resource.creator\_seq, and, giving in to current abuses discussed above,
2806 res\_role.role\_name). For all such fields, no upper limit can sensibly
2807 be defined. However, since certain DBMSes (e.g., MySQL older than
2808 version 5.6) cannot index fields with a TEXT datatype and thus using
2809 VARCHAR may be necessary at least for frequenly-searched fields, we give
2810 the maximal length of the fields in the 2013 registry in parentheses after
2811 the column designations for the TEXT datatype:
2812
2813 \begin{inlinetable}
2814 \begin{tabular}{llp{6cm}}
2815 \sptablerule
2816 \textbf{Field type}&
2817 \textbf{Datatype}&
2818 \textbf{Pertinent Fields}\\
2819 \sptablerule
2820 free string values&\texttt{TEXT}&resource.res\_description
2821 (7801)\hfil\break
2822 resource.creator\_seq (712)\hfil\break
2823 res\_role.role\_name (712)\hfil\break
2824 res\_schema.schema\_description (934)\hfil\break
2825 res\_table.table\_description (934)\hfil\break
2826 table\_column.description (3735)\hfil\break
2827 intf\_param.description (347)\hfil\break
2828 capability.cap\_description (100)\\
2829 \sptablerule
2830 titles, etc.&\texttt{VARCHAR(255)}&resource.res\_title\hfil\break
2831 res\_role.address\hfil\break
2832 res\_schema.schema\_title\hfil\break
2833 res\_table.table\_title\hfil\break
2834 relationship.related\_name\hfil\break
2835 res\_detail.detail\_value\\
2836 \sptablerule
2837 expressions&\texttt{VARCHAR(128)}&resource.version\hfil\break
2838 resource.source\_value\hfil\break
2839 res\_subject.res\_subject\\
2840 \sptablerule
2841 names&\texttt{VARCHAR(64)}&res\_table.table\_name\hfil\break
2842 table\_column.name\hfil\break
2843 intf\_param.name\\
2844 \sptablerule
2845 misc.~short strings&\texttt{VARCHAR(32)}&resource.source\_format\hfil\break
2846 res\_role.telephone\hfil\break
2847 res\_schema.schema\_name\hfil\break
2848 interface.intf\_type\hfil\break
2849 interface.intf\_role\hfil\break
2850 relationship\_type\hfil\break
2851 res\_date.value\_role\\
2852 \sptablerule
2853 misc.~particles&\texttt{VARCHAR(16)}&resource.short\_name\hfil\break
2854 table\_column.arraysize\hfil\break
2855 intf\_param.arraysize\hfil\break
2856 interface.std\_version\hfil\break
2857 intf\_param.use\\
2858 \end{tabular}
2859 \end{inlinetable}
2860
2861 % HTML section ends
2862
2863 % section appBD - RDB impl.notes
2864 % HTML section start
2865
2866 \section{XSLT to enumerate Relational
2867 Registry XPaths}
2868
2869 \label{appMkut}
2870
2871 The (non-normative) following XSL stylesheet emits xpaths as
2872 discussed in section \ref{vorutypes} when applied to a
2873 VOResource extension. Considering readability and limitations of XSLT, this is
2874 not fully general -- if VOResource extensions started to inherit from other
2875 extensions' subclasses of Resource, Capability, or Interface, this stylesheet
2876 will need to be extended.
2877
2878 Still, it is a useful tool when evaluating how to map a given extension
2879 to the relational registry.
2880
2881
2882 \lstinputlisting[language=XML,flexiblecolumns=true]{makeutypes.xslt}
2883
2884
2885 % HTML section ends
2886
2887 % section appMkut
2888 % HTML section start
2889
2890 \section{Changes from Previous Versions}
2891
2892 \label{changes}
2893
2894 \subsection{Changes from PR-2014-10-30}
2895
2896 \begin{itemize}
2897 \item No changes to specification content (only minor typo fixes).
2898 \end{itemize}
2899
2900 % HTML section start
2901
2902 \subsection{Changes from PR-20140627}
2903
2904 \begin{itemize}
2905 \item Removed reference to a future STC extension.
2906 \item Migrated to ivoatex.
2907 \end{itemize}
2908
2909 \subsection{Changes from PR-20140227}
2910
2911 \label{changes-20140227}
2912
2913
2914 \begin{itemize}
2915
2916 \item Added a \texttt{/full} details xpath from VORegistry (this had
2917 been forgotten due to limitations in the makeutypes stylesheet).{}
2918
2919 \item Added a \texttt{/capability/interface/securityMethod/@standardID}
2920 details xpath from vr:Interface.{}
2921
2922 \item Added requirement to implement the \rtent{ivo\_string\_agg}
2923 user defined function.{}
2924
2925 \item Added a section specifying the treatment of non-ASCII characters
2926 in RegTAP columns.{}
2927
2928 \item New rules on string normalization: strings must be
2929 whitespace-stripped, empty strings must be mapped to NULL.{}
2930
2931 \item Dropped requirements that the \texttt{\_index} columns are
2932 integers (let alone small integers); added a section discussing in
2933 what sense they are implementation defined.{}
2934
2935 \item Dropped adql: prefixes on TAP\_SCHEMA.columns datatypes.{}
2936
2937 \item Now declaring a precedence of xpaths generated by rules over the
2938 list in Appendix \ref{d_u_list}.{}
2939
2940 \item Clarified translation of column/@std and param/@std.{}
2941
2942 \item Now recommending to constrain on \rtent{intf\_type}
2943 (rather than \rtent{intf\_role}, as before) when locating standard
2944 interfaces.{}
2945
2946 \item Redactional changes from RFC (e.g., in column descriptions, some
2947 clarifications, typo fixes).{}
2948
2949 \end{itemize}
2950
2951 % HTML section ends
2952
2953 % section changes-20140227
2954 % HTML section start
2955
2956 \subsection{Changes from WD-20131203}
2957
2958 \label{changes-20131203}
2959
2960
2961 \begin{itemize}
2962
2963 \item Changed the data model identifier to
2964 \texttt{ivo://ivoa.net/std/RegTAP\#1.0} to match usage with a
2965 later standards record.{}
2966
2967 \item Fixed a typo in a column name: schema.schemaname is now schema.schema\_name
2968 as in the prose.{}
2969
2970 \item Recovered
2971 \texttt{/capability/uploadMethod/@ivo-id} res\_detail keys that was
2972 accidentally lost in a previous version.{}
2973
2974 \item Clarification of nomenclature.{}
2975
2976 \end{itemize}
2977
2978 % HTML section ends
2979
2980 % section changes-20131203
2981 % HTML section start
2982
2983 \subsection{Changes from WD-20130909}
2984
2985 \label{changes-20130909}
2986
2987
2988 \begin{itemize}
2989
2990 \item Updates for REC of SimpleDALRegExt, which contains versions 1.1 of
2991 both the sia and the ssap XML schemas; this means there are now additional
2992 namespace URIs to take into accound, as well as new res\_detail xpaths
2993 \texttt{/capability/maxSearchRadius},
2994 \texttt{/capability/maxImageSize}, and
2995 \texttt{/capability/testQuery/pos/refframe}.{}
2996
2997 \item Reinstated makeutypes.xslt script; it's useful even with the new
2998 xpaths.{}
2999
3000 \end{itemize}
3001
3002 % HTML section ends
3003
3004 % section changes-20130909
3005 % HTML section start
3006
3007 \subsection{Changes from WD-20130411}
3008
3009 \label{changes-20130411}
3010
3011
3012 \begin{itemize}
3013
3014 \item The final utype reform: most of our ex-utype strings aren't called utypes
3015 any more, they're fairly plain xpaths. Consequently,
3016 \rtent{res\_detail.detail\_utype} has been renamed
3017 \rtent{res\_detail.detail\_xpath}.{}
3018
3019 \item Renamed some columns and the subject table to relieve the need of quoting
3020 in MS SQL Server (or, in the case or \rtent{use\_param}, maintain
3021 consistency after the renaming):\\
3022
3023 \begin{tabular}{lll}
3024
3025 \textbf{Old}&
3026 \textbf{New}\\
3027 resource.version&resource.res\_version\\
3028 res\_role.address&res\_role.street\_address\\
3029 subject.*&res\_subject.*\\
3030 res\_subject.res\_subject&res\_subject.res\_subject\\
3031 table\_column.description&table\_column.column\_description\\
3032 intf\_param.description&intf\_param.param\_description\\
3033 intf\_param.use\_param&intf\_param.param\_use\\
3034 validation.level&validation.val\_level\\
3035
3036 \end{tabular}
3037
3038 \item rr.intf\_param grew the arraysize and delim columns that before
3039 accidentally were only present in rr.table\_column.{}
3040
3041 \item Added warnings about having to match case-insensitively in
3042 res\_detail.detail\_value for IVORN-valued rows.{}
3043
3044 \item Restored the foreign key from interface to capability. Mandating
3045 ignoring interface elements from StandardsRegExt records really is the
3046 lesser evil.{}
3047
3048 \item \rtent{resource.region\_of\_regard} now must have unit metadata
3049 declared.{}
3050
3051 \item We now explicitely deprecate multiple access URLs per interface
3052 and announce that single access URLs will be enforced in future
3053 VOResource versions.{}
3054
3055 \end{itemize}
3056
3057 % HTML section ends
3058
3059 % subsection changes-20130411
3060 % HTML section start
3061
3062 \subsection{Changes from WD-20130305}
3063
3064 \label{changes-20130305}
3065
3066 Changes from WD-20130305
3067 \begin{itemize}
3068
3069 \item intf\_index is now required to be unique within a resource, not a
3070 capability; this is because StandardsRegExt has interfaces outside
3071 of capabilities. In consequence, the intf\_param no longer has a
3072 cap\_index column, and its foreign key is just ivoid and intf\_index.{}
3073
3074 \item Added handling for the StandardsRegExt schema element.{}
3075
3076 \item The list of res\_details utypes was moved to an appendix since
3077 it was too long to be included in the running text.{}
3078
3079 \item Redation for WD publication.{}
3080
3081 \end{itemize}
3082
3083 % HTML section ends
3084
3085 % HTML section start
3086
3087 \subsection{Changes from WD-20121112}
3088
3089 \label{changes-20121112}
3090
3091 Changes from WD-20121112
3092 \begin{itemize}
3093
3094 \item Adapted all utypes to better match future VO-DML utypes.{}
3095
3096 \item footprint, data\_url, facility, and instrument are no longer in rr.resource
3097 but are instead kept in rr.res\_details rows.{}
3098
3099 \item For VOResource compliance, intf\_param has no flag column any more.{}
3100
3101 \item res\_role.base\_utype is renamed to res\_role.base\_role and no longer
3102 pretends to be a utype fragment; also, the content is now a simple
3103 word..{}
3104
3105 \item intf\_param.use is now called intf\_param.use\_param to avoid possible
3106 clashes with reserved SQL words.{}
3107
3108 \item Removed all material on STC coverage.{}
3109
3110 \item Added an appendix recommending field sizes.{}
3111
3112 \end{itemize}
3113
3114 % HTML section ends
3115
3116 % changes-20121112
3117 % HTML section ends
3118
3119
3120 % appendices
3121
3122 \bibliography{ivoatex/ivoabib,local}
3123
3124 \end{document}
3125

msdemlei@ari.uni-heidelberg.de
ViewVC Help
Powered by ViewVC 1.1.26