1 |
\documentclass[11pt,a4paper]{ivoa} |
2 |
\input tthdefs |
3 |
|
4 |
\usepackage[utf8]{inputenc} |
5 |
|
6 |
\title{Table Access Protocol} |
7 |
|
8 |
\ivoagroup{Data Access Layer Working Group} |
9 |
|
10 |
\author{Patrick Dowler} |
11 |
\author{TBD} |
12 |
|
13 |
\editor{Patrick Dowler} |
14 |
|
15 |
\previousversion[http://www.ivoa.net/Documents/TAP/1.0]{TAP-1.0} |
16 |
|
17 |
|
18 |
\begin{document} |
19 |
|
20 |
\begin{abstract} |
21 |
The table access protocol (TAP) defines a service protocol for accessing general |
22 |
table data, including astronomical catalogs as well as general database tables. |
23 |
Access is provided for both database and table metadata as well as for actual |
24 |
table data. This version of the protocol includes support for multiple query |
25 |
languages, including queries specified using the Astronomical Data Query |
26 |
Language (ADQL [1]) within an integrated interface. It also includes support |
27 |
for both synchronous and asynchronous queries. Special support is provided for |
28 |
spatially indexed queries using the spatial extensions in ADQL. A multi-position |
29 |
query capability permits queries against an arbitrarily large list of |
30 |
astronomical targets, providing a simple spatial cross-matching capability. |
31 |
More sophisticated distributed cross-matching capabilities are possible by |
32 |
orchestrating a distributed query across multiple TAP services. |
33 |
\end{abstract} |
34 |
|
35 |
|
36 |
\section*{Acknowledgments} |
37 |
|
38 |
The authors would like to acknowledge all contributors to this and previous |
39 |
versions of this standard, especially: K. Andrews, J. Good, R. Hanisch, G. |
40 |
Lemson, T. McGlynn, K. Noddle, F. Ochsenbein, I. Ortiz, P. Osuna, R. Plante, G. |
41 |
Rixon, J. Salgado, A. Stebe, and A. Szalay. |
42 |
|
43 |
|
44 |
\section*{Conformance-related definitions} |
45 |
|
46 |
The words ``MUST'', ``SHALL'', ``SHOULD'', ``MAY'', ``RECOMMENDED'', and |
47 |
``OPTIONAL'' (in upper or lower case) used in this document are to be |
48 |
interpreted as described in IETF standard, \citet{std:RFC2119}. |
49 |
|
50 |
The \emph{Virtual Observatory (VO)} is general term for a collection of |
51 |
federated resources that can be used to conduct astronomical research, |
52 |
education, and outreach. The \href{http://www.ivoa.net}{International |
53 |
Virtual Observatory Alliance (IVOA)} is a global collaboration of separately |
54 |
funded projects to develop standards and infrastructure that enable VO |
55 |
applications. |
56 |
|
57 |
|
58 |
\section{Introduction} |
59 |
The Table Access Protocol (TAP) is a Web-service protocol that gives access to |
60 |
collections of tabular data referred to collectively as a tableset. TAP |
61 |
services accept queries posed against the tableset available via the service and |
62 |
return the query response as another table, in accord with the relational model. |
63 |
Queries may be submitted using various query languages and may execute |
64 |
synchronously or asynchronously. Support for the Astronomical Data Query |
65 |
Language (ADQL, [1]) is mandatory; support for other query languages is allowed |
66 |
and optional. |
67 |
|
68 |
The result of a TAP query is another table, normally returned as a VOTable. |
69 |
Support for VOTable output is mandatory; all other formats are optional. |
70 |
|
71 |
The table collections made accessible via TAP are typically stored in relational |
72 |
database management systems (RDBMS). A TAP service exposes the database schema |
73 |
to client applications so that queries can be posed directly against arbitrary |
74 |
data tables available via the service. |
75 |
|
76 |
Multi-table operations such as joins or cross matches are possible provided the |
77 |
tables are all managed by the local TAP service, and provided the service |
78 |
supports these capabilities. Larger scale operations such as a distributed |
79 |
cross match are also possible, but require combining the results of multiple TAP |
80 |
services. |
81 |
|
82 |
\subsection{Role within the VO Architecture} |
83 |
|
84 |
NOTE: not in TAP-1.0 |
85 |
|
86 |
\begin{figure} |
87 |
\centering |
88 |
|
89 |
% Get the architecture diagram from the TCG chair |
90 |
% http://wiki.ivoa.net/twiki/bin/view/IVOA/IvoaTCG |
91 |
% If they give you a PDF, for now dumb it down to a png by |
92 |
% convert -antialias -density 72x72 archdiag.pdf archdiag.png |
93 |
% Oh -- Notes don't need this; you'd have to remove archdiag.png |
94 |
% from FIGURES in the Makefile, too. |
95 |
|
96 |
\includegraphics[width=0.9\textwidth]{archdiag.png} |
97 |
\caption{Architecture diagram for this document} |
98 |
\label{fig:archdiag} |
99 |
\end{figure} |
100 |
|
101 |
Fig.~\ref{fig:archdiag} shows the role this document plays within the |
102 |
IVOA architecture \citep{note:VOARCH}. |
103 |
|
104 |
|
105 |
\subsection{Motivating Use Cases} |
106 |
Below are some of the more common use cases that have motivated the development |
107 |
of the TAP specification. While this is not complete, it helps to understand the |
108 |
problem area covered by this specification. |
109 |
|
110 |
\subsubsection{Discover Metadata} |
111 |
Since content in relational databases is often custom and project-specific, |
112 |
users of a TAP service must be able to discover the names of tables and |
113 |
columns, datatypes, units, and other information necessary to construct |
114 |
meaningful correct queries. |
115 |
|
116 |
\subsubsection{Query Custom Tables} |
117 |
A large amount of astronomical data and metadata is stored in tables in |
118 |
relational databases. Historically, users could query these tables through |
119 |
custom user interfaces (usually web page forms), but such approaches could not |
120 |
provide support for truly ad-hoc querying. A TAP service should enable users to |
121 |
discover and query custom tables with a flexible and expressive input that |
122 |
supports ad-hoc querying: selecting output, filtering the result, joining |
123 |
multiple tables, computing aggregate quantities, etc. |
124 |
|
125 |
\subsubsection{Query Standard Tables} |
126 |
A TAP service should enable users to query externally defined standard tables |
127 |
in a uniform way such that the same web service request can be sent to multiple |
128 |
services. Services must be able to declare their support for standard tables in |
129 |
the service metadata. |
130 |
|
131 |
\subsubsection{Query Standard Data Models} |
132 |
A TAP service should enable users to query (parts of) externally defined data |
133 |
models that are (partially or fully) implemented by the service. Services must |
134 |
be able to declare their support for data models as well as the way that model |
135 |
elements are mapped to tables and columns. |
136 |
|
137 |
\subsubsection{ADQL Queries} |
138 |
The Astronomical Data Query Language (ADQL) [std:ADQL] is the standard |
139 |
query language for the IVOA. Support for ADQL queries is mandatory. ADQL can be |
140 |
used to specify queries that access one or more tables provided by the TAP |
141 |
service, including the standard metadata tables. In general, the client must |
142 |
access table metadata in order to discover the names of tables and columns and |
143 |
then formulate queries. ADQL queries provide a direct (low-level) access to the |
144 |
tables; a query will be written for a specific TAP service and will not be |
145 |
usable with other services unless the query refers only to common tables and |
146 |
columns. It is also possible that the service registration (in an IVOA Registry) |
147 |
may include sufficient table metadata to enable queries to be written directly. |
148 |
|
149 |
\subsubsection{Other Query Languages} |
150 |
A TAP service must be able to support use of other query languages, such |
151 |
pass-through of native SQL directly to an underlying DBMS or simple key-vale |
152 |
(parameter-based) constraints. The service interface must allow for |
153 |
this and the service capabilities must be able to describe it. This mechanism |
154 |
also allows future developments within and outside the IVOA to be used without |
155 |
revising the TAP specification. |
156 |
|
157 |
\subsubsection{Asynchronous Queries} |
158 |
Asynchronous queries allow for long running queries to complete without |
159 |
the client maintaining a connection to the service. Results are stored by |
160 |
the service for later retrieval by the client. Asynchronous query |
161 |
execution is generally more robust and not susceptible to time-outs or other |
162 |
transient failures. They are especially suited to queries that run for a long |
163 |
time before producing output (e.g. queries that compute or aggregate values). |
164 |
|
165 |
\subsubsection{Synchronous Queries} |
166 |
Synchronous queries execute immediately and the client must wait for the query |
167 |
to finish. Synchronous query execution is generally simpler and provides a |
168 |
faster (low latency) response and should be adequate when the query will execute |
169 |
and start returning results quickly. Even with large query results, synchronous |
170 |
queries are a good approach as long as the service can stream the output |
171 |
and consume modest internal resources. |
172 |
|
173 |
|
174 |
\subsection{Interface Overview -- move to examples section?} |
175 |
Table Access Protocol (TAP) is implemented over the HTTP protocol using standard |
176 |
HTTP GET and POST requests and conventions. A TAP request specifies one or more |
177 |
parameter key/value pairs; both keys and values are strings. The keys used are |
178 |
discussed in this specification and in the specifications for query languages |
179 |
supported by a service. The values may need to be encoded, using standard |
180 |
URL-encoding. For the following examples, http://example.com/tap/ is the base |
181 |
URL for a TAP service. |
182 |
|
183 |
This is an example of a synchronous ADQL query on r magnitude: |
184 |
|
185 |
\begin{verbatim} |
186 |
HTTP POST htp://example.com/tap/sync |
187 |
REQUEST=doQuery |
188 |
LANG=ADQL |
189 |
QUERY=SELECT * FROM magnitudes as m where m.r>=10 and m.r<=16 |
190 |
\end{verbatim} |
191 |
|
192 |
Synchronous queries return the table of results in the HTTP response to the |
193 |
initial request. In the examples above, the output format defaults to VOTable; |
194 |
the FORMAT parameter could be added to select a different format. |
195 |
|
196 |
Asynchronous queries are created in the same way as the synchronous kind, using |
197 |
the /async endpoint: |
198 |
|
199 |
\begin{verbatim} |
200 |
HTTP POST http://example.com/tap/async |
201 |
REQUEST=doQuery |
202 |
LANG=ADQL |
203 |
QUERY=SELECT * FROM magnitudes AS m WHERE m.r>=10 AND m.r<=16 |
204 |
\end{verbatim} |
205 |
|
206 |
The service's response to these requests is a URL representing the query's |
207 |
state and progress and where the state may be monitored and controlled. The |
208 |
query |
209 |
result or an error document can then be retrieved from a URL associated with the |
210 |
job. This is an application of the UWS pattern. The query is then executed |
211 |
with a separate request to run the job URL: |
212 |
|
213 |
\begin{verbatim} |
214 |
HTTP POST http://example.com/tap/async/<jobid>/phase |
215 |
PHASE=RUN |
216 |
\end{verbatim} |
217 |
|
218 |
The state of the job can be retrieved from the phase resource: |
219 |
|
220 |
\begin{verbatim} |
221 |
HTTP GET http://example.com/tap/async/<jobid>/phase |
222 |
\end{verbatim} |
223 |
|
224 |
The client may have to check the phase multiple times until the job |
225 |
finishes. Once the returned value is COMPLETED, the results can be obtained |
226 |
from the results resource: |
227 |
|
228 |
\begin{verbatim} |
229 |
HTTP GET http://example.com/tap/async/<jobid>/results/result |
230 |
\end{verbatim} |
231 |
|
232 |
In addition to the sync and async resources for query execution, a TAP service |
233 |
also has metadata resources defined by the VOSI standard. The availability of a |
234 |
service can be monitored by accessing: |
235 |
|
236 |
\begin{verbatim} |
237 |
HTTP GET http://example.com/tap/availability |
238 |
\end{verbatim} |
239 |
|
240 |
The complete table metadata can be obtained: |
241 |
|
242 |
\begin{verbatim} |
243 |
HTTP GET http://example.com/tap/tables |
244 |
\end{verbatim} |
245 |
|
246 |
The capabilities can be obtained by: |
247 |
|
248 |
\begin{verbatim} |
249 |
HTTP GET http://example.com/tap/capabilities |
250 |
\end{verbatim} |
251 |
|
252 |
The capabilities are also accessible via a service request to the synchronous |
253 |
query resource: |
254 |
|
255 |
\begin{verbatim} |
256 |
HTTP GET http://example.com/tap/sync?REQUEST=getCapabilities |
257 |
\end{verbatim} |
258 |
|
259 |
This output lists support for optional TAP functionality and additional |
260 |
implemented interfaces. |
261 |
|
262 |
\section{Resources} |
263 |
|
264 |
An implementation of a TAP service provides the following RESTful resources |
265 |
under the base URL. |
266 |
|
267 |
\begin{tabular}{l l l l l} |
268 |
resource type & resource name & required \\ |
269 |
{sync} & /sync & must (anonymous) & \\ |
270 |
{async} & /async & must (anonymous) & \\ |
271 |
{sync} & service specific & may (alternate authentication method) & \\ |
272 |
{async} & service specific & may (alternate authentication method) & \\ |
273 |
VOSI-availability & /availability & should & \\ |
274 |
VOSI-capabilities & /capabilities & must & \\ |
275 |
VOSI-tables & /tables & should & \\ |
276 |
DALI-examples & /examples & should & \\ |
277 |
\end{tabular} |
278 |
|
279 |
At least one set of {sync} and {async} resources must be named /sync and |
280 |
/async respectively for backwards compatibility with TAP-1.0 (which required |
281 |
these names. These resources must be used for anonymous query execution. Other |
282 |
{sync} and {async} resources may have service specific names, but all resources |
283 |
listed above must be siblings so that a client with one such URL can find the |
284 |
VOSI-capabilities URL and thus discover other available resources. |
285 |
|
286 |
A TAP service must be represented as a set of sibling web resources each |
287 |
addressable via a URL in the http scheme, or the https scheme, or both. |
288 |
|
289 |
The web resource at the root of the tree must represent the service as a whole. |
290 |
This specification defines no standard representation for this root resource. |
291 |
Implementations may provide a representation, or may return a '404 not found' |
292 |
response to requests for the root web-resource. One possible representation is |
293 |
an HTML page describing the scientific usage and content of the service. TAP |
294 |
clients must not depend on a specific representation of the root web-resource. |
295 |
|
296 |
\subsection{\{sync\}} |
297 |
A TAP service must provide a web resource with relative URL /sync that is a |
298 |
direct child of the root web resource. This web resource represents the results |
299 |
of synchronous requests. The exact form of the query, and hence the |
300 |
representation of the resource, is defined by the query parameters as listed in |
301 |
section 2.3. Representations of results of queries and VOSI outputs are defined |
302 |
in sections 2.7.1 and 2.7.2 respectively. |
303 |
|
304 |
For query languages that produce a single result (e.g. ADQL) executed using the |
305 |
/sync endpoint, the result of a successful query is returned in the response or |
306 |
the response includes an HTTP redirect (303: See Other) to a resource (uri) from |
307 |
which the result may be retrieved. |
308 |
|
309 |
An HTTP-GET request to the /sync web resource may return a cached copy of the |
310 |
representation. This cached copy might come from an HTTP cache between the |
311 |
client and the service, and the service may also maintain its own cache. Clients |
312 |
which require an up-to-date representation of volatile data or metadata must use |
313 |
HTTP POST. |
314 |
|
315 |
\subsection{\{async\}} |
316 |
A TAP service must provide a web resource with relative URL /async that is a |
317 |
direct child of the root web resource. This web resource represents controls for |
318 |
asynchronous queries. Specifically, the web resource must represent the job-list |
319 |
as specified in the UWS standard [3]. |
320 |
|
321 |
The child web resources of the /async resource are as specified by UWS. These |
322 |
are descendants of the /async web-resource, and they include a web resource that |
323 |
represents the eventual result of an asynchronous query, e.g.: |
324 |
\begin{verbatim} |
325 |
http://example.com/tap/async/42/results/result |
326 |
\end{verbatim} |
327 |
where the base URL for the TAP service is: |
328 |
\begin{verbatim} |
329 |
http://example.com/tap |
330 |
\end{verbatim} |
331 |
the UWS job list is: |
332 |
\begin{verbatim} |
333 |
http://example.com/tap/async |
334 |
\end{verbatim} |
335 |
and the job resource is |
336 |
\begin{verbatim} |
337 |
http://example.com/tap/async/42 |
338 |
\end{verbatim} |
339 |
where 42 is the job identifier. A client making an asynchronous request must use |
340 |
the UWS facilities to monitor or control the job. In addition to the job list |
341 |
and job resource above, UWS specifies the name and semantics of the a small set |
342 |
of child resources used to view and control the job, e.g.: |
343 |
\begin{verbatim} |
344 |
http://example.com/tap/async/42/phase |
345 |
http://example.com/tap/async/42/quote |
346 |
http://example.com/tap/async/42/executionduration |
347 |
http://example.com/tap/async/42/destruction |
348 |
http://example.com/tap/async/42/error |
349 |
http://example.com/tap/async/42/parameters |
350 |
http://example.com/tap/async/42/results |
351 |
http://example.com/tap/async/42/owner |
352 |
\end{verbatim} |
353 |
Successful TAP queries produce results which must be accessible as resources |
354 |
under the UWS result list, e.g.: |
355 |
\begin{verbatim} |
356 |
http://example.com/tap/async/42/results/ |
357 |
\end{verbatim} |
358 |
Failed TAP queries produce an error document (see 2.9 ) which must be accessible |
359 |
as the error resource, e.g.: |
360 |
\begin{verbatim} |
361 |
http://example.com/tap/async/42/error |
362 |
\end{verbatim} |
363 |
For query languages that produce a single result executed using the /async |
364 |
endpoint, the result of a successful query can be found within the result list |
365 |
specified by UWS [3]; the result must be named result and thus clients are able |
366 |
to access it directly, e.g.: |
367 |
\begin{verbatim} |
368 |
http://example.com/tap/async/42/results/result |
369 |
\end{verbatim} |
370 |
Access of this resource must deliver the result, either directly or as an HTTP |
371 |
redirect (303: See Other) to a resource from which the result may be retrieved. |
372 |
|
373 |
For query languages that may produce multiple result resources, the names of the |
374 |
results are not specified (they may be specified in the specification for the |
375 |
language). The client can always access the result list resource as specified by |
376 |
UWS [3]. |
377 |
|
378 |
If the query returned no rows, the result resource must exist and contain no |
379 |
data rows. Details on interacting with these resources are specified in the UWS |
380 |
standard; for examples specific to TAP see Section 5 below. |
381 |
|
382 |
\subsection{/availability} |
383 |
The VOSI availability metadata should be accessible from a web resource with |
384 |
relative URL /availability that is a direct child of the root web resource. If |
385 |
implemented, the /availability resource must be accessible via the http GET |
386 |
method. The content is described by [6]. |
387 |
|
388 |
Services which do not implement the /availability resource must respond with an |
389 |
HTTP response code of 404 when this resource is accessed. |
390 |
|
391 |
\subsection{/capabilities} |
392 |
The TAP-1.0 standard is identified using |
393 |
\begin{verbatim} |
394 |
ivo://ivoa.net/std/TAP |
395 |
\end{verbatim} |
396 |
|
397 |
For TAP-1.1 we define new standard identifiers for each of the |
398 |
features. Asynchronous query execution is described by standardID: |
399 |
|
400 |
\begin{verbatim} |
401 |
ivo://ivoa.net/std/TAP#async-1.1 |
402 |
\end{verbatim} |
403 |
|
404 |
Synchronous query execution is described by standardID: |
405 |
|
406 |
\begin{verbatim} |
407 |
standardID="ivo://ivoa.net/std/TAP#sync-1.1 |
408 |
\end{verbatim} |
409 |
|
410 |
In TAP-1.0 the base URL was described with a single standard identifier; in |
411 |
TAP-1.1 and beyond, individual resources are described with their on |
412 |
standardID. This allows service providers to describe multiple resources that |
413 |
deliver the specified feature (e.g with different authentication methods) in |
414 |
the VOSI-capabilities resource. |
415 |
|
416 |
The VOSI standard specifies that the capability metadata is encoded as an XML |
417 |
document which lists each of the service's capabilities as a <capability> |
418 |
element. The type of this element (which defines the contents) is |
419 |
{http://www.ivoa.net/xml/VOResource/v1.0}Capability from the VOResource XML |
420 |
standard [8]. |
421 |
|
422 |
In addition, the capabilities output must also comply with the following |
423 |
requirements: |
424 |
|
425 |
* the returned document must include <capability> elements |
426 |
that describes the service's support for the TAP protocol using the TAP and |
427 |
VOSI standardID values |
428 |
|
429 |
* this capability element must include at least one <interface> element |
430 |
with its "role" attribute set to "std", |
431 |
|
432 |
Note: VO registries recognize a service's support for a standard protocol |
433 |
through this capability description. In particular, a different standard |
434 |
Capability sub-type is used for each standard protocol to provide capability |
435 |
metadata that is specific to that protocol. At the time of this writing, a |
436 |
Capability sub-type for TAP has not yet been defined. Thus for compliance with |
437 |
this standard, any legal Capability description that meets the above |
438 |
restrictions is sufficient. However, once a VOResource extension for TAP is |
439 |
standardized, it is strongly recommended that TAP services emit its |
440 |
capabilities using that the Capability sub-type specialized for TAP. |
441 |
|
442 |
For example, the returned capabilities document for a service supporting TAP |
443 |
might look as follows: |
444 |
|
445 |
\begin{verbatim} |
446 |
<?xml version="1.0" encoding="UTF-8"?> |
447 |
|
448 |
<vosi:capabilities xmlns="" |
449 |
xmlns:vosi="http://www.ivoa.net/xml/VOSI/v1.0" |
450 |
xmlns:vs="http://www.ivoa.net/xml/VODataService/v1.0" |
451 |
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" |
452 |
xsi:schemaLocation="http://www.ivoa.net/xml/VOSI/v1.0 |
453 |
http://www.ivoa.net/xml/VOSI/v1.0 |
454 |
http://www.ivoa.net/xml/VODataService/v1.0 |
455 |
http://www.ivoa.net/xml/VODataService/v1.0"> |
456 |
|
457 |
<vosi:capability standardID="ivo://ivoa.net/std/TAP#async-1.1"> |
458 |
<interface xsi:type="vs:ParamHTTP" role="std"> |
459 |
<accessURL use="base"> http://myarchive.net/myTAP/async </accessURL> |
460 |
</interface> |
461 |
</vosi:capability> |
462 |
|
463 |
<vosi:capability standardID="ivo://ivoa.net/std/TAP#sync-1.1"> |
464 |
<interface xsi:type="vs:ParamHTTP" role="std"> |
465 |
<accessURL use="base"> http://myarchive.net/myTAP/sync </accessURL> |
466 |
</interface> |
467 |
</vosi:capability> |
468 |
|
469 |
<vosi:capability standardID="ivo://ivoa.net/std/VOSI#capabilities"> |
470 |
<interface xsi:type="vs:ParamHTTP"> |
471 |
<accessURL use="full"> |
472 |
http://myarchive.net/myTAP/capabilities </accessURL> |
473 |
</interface> |
474 |
</vosi:capability> |
475 |
|
476 |
<vosi:capability standardID="ivo://ivoa.net/std/VOSI#availability"> |
477 |
<interface xsi:type="vs:ParamHTTP"> |
478 |
<accessURL use="full"> |
479 |
http://myarchive.net/myTAP/availability |
480 |
</accessURL> |
481 |
</interface> |
482 |
</vosi:capability> |
483 |
|
484 |
<vosi:capability standardID="ivo://ivoa.net/std/VOSI#tables"> |
485 |
<interface xsi:type="vs:ParamHTTP"> |
486 |
<accessURL use="full"> |
487 |
http://myarchive.net/myTAP/tables </accessURL> |
488 |
</interface> |
489 |
</vosi:capability> |
490 |
|
491 |
</vosi:capabilities> |
492 |
\end{verbatim} |
493 |
|
494 |
The service capabilities must be accessible from a web resource with relative |
495 |
URL /capabilities that is a direct child of the root web resource. The |
496 |
/capabilities resource must be accessible via the http GET method. The content |
497 |
is described by [8]. |
498 |
|
499 |
\subsection{/tables} |
500 |
The table metadata should be accessible from a web resource with relative URL |
501 |
/tables that is a direct child of the root web resource. The /tables resource |
502 |
must be accessible via the http GET method. The content is described by [7] and |
503 |
is equivalent to the metadata from the TAP\underline{' '}SCHEMA described in |
504 |
XREF. |
505 |
|
506 |
Services which do not implement the /tables resource must respond with an HTTP |
507 |
response code of 404 when this resource is accessed. |
508 |
|
509 |
\subsection{/examples} |
510 |
A GET from this endpoint MUST yield a document with a MIME type of either |
511 |
application/xhtml+xml or text/html. A service that does not provide examples |
512 |
MUST return a 404 HTTP status on accessing this resource. |
513 |
|
514 |
If present, the endpoint must be represented in a capability in the TAP |
515 |
service's registry record. The capability's standardID is defined by |
516 |
[std:DALI]. A capability element could hence look like this: |
517 |
|
518 |
\begin{verbatim} |
519 |
<capability standardID="ivo://ivoa.net/std/DALI#examples"> |
520 |
<interface xsi:type="vr:WebBrowser"> |
521 |
<accessURL use="full">http://myarchive.net/myTAP/examples</accessURL> |
522 |
</interface> |
523 |
</capability> |
524 |
\end{verbatim} |
525 |
|
526 |
TAP defines two additional properties vocabulary: |
527 |
|
528 |
* query -- each example MUST have a unique child element with simple text |
529 |
content having a property attribute valued query. It contains the query itself, |
530 |
preferably with extra whitespace for easy human consumption and editing. This |
531 |
will usually be a HTML pre element. |
532 |
|
533 |
* table -- examples MAY also have descendants with property attributes having |
534 |
the value table. These must have pure text content and contain fully qualified |
535 |
table names to which the query is somehow "pertaining". Suitable HTML elements |
536 |
holding these include span, or a (which would allow linking to further |
537 |
information on the table). |
538 |
|
539 |
When using elements with src or href attributes to carry the property |
540 |
attributes, note that the element content must be repeated in a content |
541 |
attribute, as otherwise RDFa clients would interpret the embedded link rather |
542 |
than the element content as the object in the triple. |
543 |
|
544 |
TODO: add example(s) here |
545 |
|
546 |
\subsection{Parameters for \{async\} and \{sync\}} |
547 |
The \{async\} and \{sync\} web-resources must accept the parameters listed in |
548 |
the following sub-sections. In a synchronous request, the parameters select the |
549 |
representation returned in the response message. In an asynchronous request, the |
550 |
parameters select the representation of the eventual query result rather than |
551 |
the response to the initial request. |
552 |
|
553 |
Requirements on the presence and values of parameters described below are |
554 |
enforced only when the TAP request is executed (not when individual HTTP |
555 |
requests are handled). Thus, for asynchronous TAP queries, the parameter |
556 |
requirements must be satisfied (and errors returned if not) only when the query |
557 |
is run in (in the sense of UWS job execution). Specifically, asynchronous |
558 |
queries may be created with with no parameters and multiple, subsequent HTTP |
559 |
POST actions may specify the parameters in any order. |
560 |
|
561 |
Not all combinations of the parameters are meaningful. For example, if a request |
562 |
carries LANG=ADQL then the SELECT parameter (from PQL) is spurious. If a |
563 |
service receives a spurious parameter in an otherwise correct request, then the |
564 |
service must ignore the spurious parameter, must respond to the request normally |
565 |
and must not report errors concerning the spurious parameter. |
566 |
|
567 |
\subsubsection{REQUEST - remove? } |
568 |
This parameter distinguishes current service operations, makes it possible to |
569 |
extend the service specification (with additional or custom operations), and |
570 |
specifies how other parameters should be interpreted. If a TAP service attempts |
571 |
to execute a TAP request without this parameter or with an incorrect value for |
572 |
this parameter, then the service must reject the request and return an error |
573 |
document as the result. |
574 |
|
575 |
These are the standard values of the parameter: |
576 |
|
577 |
doQuery: execute a query |
578 |
|
579 |
getCapabilities: return VOSI-capabilities metadata |
580 |
|
581 |
All requests to execute a query using a query language must |
582 |
include REQUEST=doQuery and must include the LANG parameter. For other values of |
583 |
REQUEST, additional parameters may or may not be required. The |
584 |
REQUEST=getCapabilities service operation must be supported for synchronous |
585 |
(/sync) requests and is not defined for asynchronous (/async) requests. |
586 |
|
587 |
For synchronous queries, the HTTP request must also include additional |
588 |
parameters (see below) with the details of the query. These are used for |
589 |
metadata queries and data queries. |
590 |
|
591 |
For asynchronous queries, the additional parameters may be included with the |
592 |
HTTP request that creates the query (the UWS job) or they may be POSTed directly |
593 |
to the created job resource, in one or more separate HTTP requests. The |
594 |
parameter names remain the same in both cases. |
595 |
|
596 |
\subsubsection{LANG} |
597 |
The LANG parameter specifies the query language. The service must support LANG |
598 |
and the client must provide a value with REQUEST=doQuery. The only standard |
599 |
values for the LANG parameter is ADQL (a required language). Support for other |
600 |
languages and the LANG value to use with them is described in the service |
601 |
capabilities. |
602 |
|
603 |
For example, an ADQL query would be performed with |
604 |
\begin{verbatim} |
605 |
REQUEST=doQuery |
606 |
LANG=ADQL |
607 |
QUERY=<ADQL query string> |
608 |
\end{verbatim} |
609 |
A PQL query would be performed with |
610 |
\begin{verbatim} |
611 |
REQUEST=doQuery |
612 |
LANG=PQL |
613 |
<PQL-specific parameters> |
614 |
\end{verbatim} |
615 |
The value of LANG is a string specifying the language and optionally the |
616 |
language version used for the query parameter(s), as defined by the service |
617 |
capabilities. The client may specify the version of the query language, e.g. |
618 |
LANG=ADQL-2.0 (the syntax should be as shown) or it may omit the version, e.g. |
619 |
LANG=ADQL. The service should return an “unknown query language” error as |
620 |
described in 2.9 if an unsupported language or an incompatible language version |
621 |
is specified. |
622 |
|
623 |
\subsubsection{QUERY} |
624 |
The QUERY parameter is used to specify the ADQL query. It may also be used to |
625 |
specify the query for other values of LANG (e.g. LANG=<some RDBMS-specific SQL |
626 |
variant>) which are not specified in this document but may be described in the |
627 |
service capabilities. |
628 |
|
629 |
A service must support the QUERY parameter because ADQL is a required language. |
630 |
The case sensitivity of the query string is defined solely by the query language |
631 |
specification. In the case of ADQL 2.0, for example, the query is not case |
632 |
sensitive except for character literals; schema, table, and column names, |
633 |
function names, and other ADQL keywords are not case sensitive. |
634 |
|
635 |
Within the ADQL query, the service must support the use of timestamp values as |
636 |
described in [std:DALI]. |
637 |
|
638 |
If the tables that are queried through a service contain columns with spatial |
639 |
coordinates and the service supports spatial querying via the ADQL “region” |
640 |
constructs, the service must support the INTERSECTS function and it must support |
641 |
the following geometry functions: REGION, POINT, BOX, CIRCLE, COORD1, COORD2, |
642 |
COORDSYS. Support for the AREA, CONTAINS, and POLYGON functions are optional. If |
643 |
the service supports the REGION function, it must support region encoding in |
644 |
STC-S format (see section 6 ); the extent of STC-S support within the REGION |
645 |
function is left up to the implementation. Coordinate system specification for |
646 |
POINT, BOX, CIRCLE, and POLYGON must use values from STC-S as described in |
647 |
section 6 . |
648 |
|
649 |
Note: Although it is allowed by the ADQL syntax, clients should be careful when |
650 |
mixing constants and column references for coordinate system and coordinate |
651 |
values. For example, POINT('ICRS', t.ra, t.dec) does not cause t.ra and t.dec to |
652 |
be transformed to ICRS; it simply tells the service to treat the values as |
653 |
being expressed in that coordinate system. |
654 |
|
655 |
\subsubsection{FORMAT and RESPONSEFORMAT} |
656 |
The RESPONSEFORMAT parameter is fully described in [std:DALI]. For backwards |
657 |
compatibility, TAP-1.1 must also accept the FORMAT parameter as equivalent to |
658 |
RESPONSEFORMAT. |
659 |
|
660 |
\subsubsection{MAXREC} |
661 |
The MAXREC parameter and its effect on the query result is fully described in |
662 |
[std:DALI]. If the result set is truncated in this fashion, it must include an |
663 |
overflow indicator as specified in section XREF. |
664 |
|
665 |
For the special value of MAXREC=0, the service is not required to execute the |
666 |
query; a successful MAXREC=0 request does not necessarily mean that the query |
667 |
is valid and the overflow indicator does not necessarily mean that there is at |
668 |
least one row satisfying the query. The service may perform validation and may |
669 |
try to execute the query, in which case a MAXREC=0 request can fail. A query |
670 |
with MAXREC=0 can be used with a simple query (e.g. SELECT * FROM |
671 |
some\underline{' '}table) to extract and examine the VOTable metadata (assuming |
672 |
FORMAT=votable). Note: in this version of TAP, this is the only mechanism to |
673 |
learn some of the detailed metadata, such as coordinate systems used. |
674 |
|
675 |
\subsubsection{RUNID} |
676 |
The RUNID parameter is fully described in [std:DALI]. |
677 |
|
678 |
\subsubsection{VERSION} |
679 |
The VERSION parameter is fully described in {std:DALI]. |
680 |
|
681 |
\subsubsection{UPLOAD} |
682 |
The UPLOAD parameter is described in [std:DALI]. Services should support the |
683 |
upload of temporary tables (in [std:VOTable] format) via the standard UPLOAD |
684 |
parameter. The table-name(s) must be legal ADQL table names as defined in |
685 |
[std:ADQL] but restricted as described below XREF. URIs maybe be simple URLs |
686 |
(e.g. with a URI scheme of http) or URIs (e.g. with a URI scheme of vos or |
687 |
param) that must be resolved to give the location of the table content. See |
688 |
section XREF for details. |
689 |
|
690 |
If table upload supported, the service must accept tables in VOTable format. |
691 |
The client specifies the name of the uploaded table; this name must be a legal |
692 |
ADQL table name with no catalog or schema (i.e., a string following the |
693 |
regular identifier production of [std:ADQL]). Uploaded tables must be referred |
694 |
to in queries as TAP\underline{' '}UPLOAD.<tablename>, where <tablename> is the |
695 |
specified by the user. Tables in the TAP\underline{' '}UPLOAD schema are |
696 |
transient and persist only for the lifetime of the query (although caching might |
697 |
be used behind the scenes) and are never visible in the |
698 |
TAP\underline{' '}SCHEMA metadata. |
699 |
|
700 |
The [std:DALI] UPLOAD parameter supports both external resources and in-line |
701 |
content. For external resources, one provides a URI (usually an HTTP URL) the |
702 |
TAP service can use to obtain the table content. For example, |
703 |
\begin{verbatim} |
704 |
HTTP POST http://example.com/tap/async/42 |
705 |
UPLOAD=mytable,http://otherplace.com/path/votable.xml |
706 |
\end{verbatim} |
707 |
The service would retrieve the table from the provided URL and |
708 |
make it visible to the query as TAP\underline{' '}UPLOAD.mytable. |
709 |
|
710 |
If the TAP service supports VOSpace (TBD: how to declare this?), one may |
711 |
specify an upload table using a URI to a table stored in a VOSpace, for example: |
712 |
\begin{verbatim} |
713 |
HTTP POST http://example.com/tap/async/42 |
714 |
UPLOAD=mytable,vos://space/path/votable.xml |
715 |
\end{verbatim} |
716 |
The service would resolve the URI, contact the VOSpace, retrieve the table, and |
717 |
make it visible to the query as TAP\underline{' '}UPLOAD.mytable. |
718 |
|
719 |
UPLOADs are accumulating, i.e., each UPLOAD parameter given will create one or |
720 |
more tables in TAP\underline{' '}UPLOAD. When the table names from two or more |
721 |
upload items agree after case folding, the service behaviour is unspecified. |
722 |
Clients thus cannot reliably overwrite uploaded tables; to correct errors, they |
723 |
have to tear down the existing job and create a new one. In principle, any |
724 |
number of tables can be uploaded using the UPLOAD parameter and any combination |
725 |
of URI schemes supported by the service as long as they are assigned unique |
726 |
table names within the query. Services may limit the size and number of |
727 |
uploaded tables; if the service refuses to accept the entire table it must |
728 |
respond with an error as described in 2.7.3 . |
729 |
|
730 |
|
731 |
\section{Use of VOTable} |
732 |
The [std:VOTable] format is the standard format for output (query results) and |
733 |
input (table upload) in a TAP service. |
734 |
|
735 |
|
736 |
\subsection{INFO elements} |
737 |
The RESOURCE element must contain an INFO element with attribute |
738 |
name="QUERY\underline{' '}STATUS" indicating the success of the operation. For |
739 |
RESOURCE elements that contain a TABLE element, this INFO element must appear |
740 |
lexically before the TABLE. The following values are defined for this INFO |
741 |
element's value attribute: |
742 |
|
743 |
“OK”, meaning that the query executed successfully and a result table is |
744 |
included in the resource |
745 |
|
746 |
"ERROR”, meaning that an error was detected at the level of the TAP |
747 |
protocol or the query failed to execute |
748 |
|
749 |
The content of the INFO element conveying the status should be a message |
750 |
suitable for display to the user describing the status. |
751 |
|
752 |
\begin{verbatim} |
753 |
<INFO name="QUERY_STATUS" value="OK"/> |
754 |
\end{verbatim} |
755 |
|
756 |
\begin{verbatim} |
757 |
<INFO name="QUERY_STATUS" value="OK">Successful query</INFO> |
758 |
\end{verbatim} |
759 |
|
760 |
\begin{verbatim} |
761 |
<INFO name="QUERY_STATUS" value="ERROR"> |
762 |
value out of range in POS=45,91 |
763 |
</INFO> |
764 |
\end{verbatim} |
765 |
|
766 |
Additional INFO elements may be provided, e.g., to echo the input parameters |
767 |
back to the client in the query response (a useful feature for debugging or to |
768 |
self-document the query response), but clients should not depend on these. |
769 |
|
770 |
\begin{verbatim} |
771 |
<RESOURCE type=”results”> |
772 |
<INFO name="QUERY_STATUS" value="ERROR"> |
773 |
unrecognized operation |
774 |
</INFO> |
775 |
<INFO name="SPECIFICATION" value="TAP"/> |
776 |
<INFO name=”VERSION” value=”1.0”/> |
777 |
<INFO name="REQUEST" value="doQuery"/> |
778 |
<INFO name="baseUrl" value="http://webtest.aoc.nrao.edu/ivoa-dal"/> |
779 |
<INFO name="serviceVersion" value="1.0"/ |
780 |
... |
781 |
</RESOURCE> |
782 |
\end{verbatim} |
783 |
|
784 |
If an overflow occurs (result exceeds MAXREC), the service must close the table |
785 |
and append another INFO element to the RESOURCE (after the TABLE) with |
786 |
name=”QUERY\underline{' '}STATUS” and the value=”OVERFLOW”. |
787 |
\begin{verbatim} |
788 |
<RESOURCE type=”results”> |
789 |
<INFO name="QUERY_STATUS" value="OK"/> |
790 |
... |
791 |
<TABLE>...</TABLE> |
792 |
<INFO name="QUERY_STATUS" value="OVERFLOW"/> |
793 |
</RESOURCE> |
794 |
\end{verbatim} |
795 |
|
796 |
In the above example, the TABLE should have exactly MAXREC rows. |
797 |
|
798 |
If an error occurs while writing the rows of the VOTable, the service must |
799 |
close the table and append another INFO element to the RESOURCE, after the |
800 |
TABLE, with name=”QUERY\underline{' '}STATUS” and the value=”ERROR”. |
801 |
\begin{verbatim} |
802 |
<RESOURCE type=”results”> |
803 |
<INFO name="QUERY_STATUS" value="OK"/> |
804 |
... |
805 |
<TABLE>...</TABLE> |
806 |
<INFO name="QUERY_STATUS" value="ERROR" /> |
807 |
</RESOURCE> |
808 |
\end{verbatim} |
809 |
The content of these trailing INFO elements is optional and intended for users; |
810 |
client software should not depend on it. |
811 |
|
812 |
Thus, one INFO element with name=”QUERY\underline{' '}STATUS” and value=”OK” or |
813 |
value=”ERROR” must be included before the TABLE. If the TABLE does not contain |
814 |
the entire query result, one INFO element with value=”OVERFLOW” or |
815 |
value=”ERROR” |
816 |
must be included after the table. |
817 |
2.9.2 Version Mismatch Errors |
818 |
|
819 |
Errors due to version mismatch from either the VERSION parameter (TAP version) |
820 |
or specific version used in the LANG parameter (query language version) are |
821 |
specified using an INFO element with name=”QUERY\underline{' '}STATUS” and |
822 |
value=”ERROR” as described above. |
823 |
|
824 |
\subsection{Successful Queries} |
825 |
|
826 |
The result of a query depends on the query language used and may be one or more |
827 |
tables in one or more resources. Unsupportable combinations of query result and |
828 |
FORMAT (e.g. queries that produce multiple tables and an inherently |
829 |
single-table format like CSV) will cause the request to fail. Currently, an ADQL |
830 |
query result must be a single table (in a single file). |
831 |
|
832 |
The output table must include the same number and order of columns as specified |
833 |
in the SELECT clause of the query. For VOTable output, the name attribute of |
834 |
FIELD elements must be the same as the column names (or aliases if specified in |
835 |
the query) from the query and the datatype, arraysize, and xtype attributes of |
836 |
FIELD elements must be set using the mapping specified in section 2.5 . The |
837 |
xtype attribute in the output must match the datatype for the column in the |
838 |
TAP\underline{' '}SCHEMA. |
839 |
|
840 |
VOTable structure follows the rules in section 2.9 and must be returned with an |
841 |
allowed VOTable MIME type (application/x-votable+xml or text/xml). If the |
842 |
FORMAT parameter (see 2.3.6 ) of the request specified a specific VOTable MIME |
843 |
type, the requested MIME type must be used in the HTTP response. |
844 |
|
845 |
CSV formatted data should represent the output table with one row of text per |
846 |
table row, with the table column values rendered as text and separated by |
847 |
commas. If a column value contains a comma the entire column value should be |
848 |
enclosed in double quotes. Text lines may be arbitrarily long. The first data |
849 |
row should give the column name as the data value. CSV data must be returned |
850 |
with a MIME type of text/csv; if the optional header line (with column names) |
851 |
is included, the MIME type must be text/csv;header=present. Full details of CSV |
852 |
format are defined in RFC 4180 [14]. |
853 |
|
854 |
TSV formatted data should represent the output table with one row of text per |
855 |
table row, with the table column values rendered as text and separated by the |
856 |
TAB character. TSV data must be returned with a MIME type of |
857 |
text/tab-separated-values [15]. Column values may not contain the TAB |
858 |
character. |
859 |
|
860 |
\subsection{Errors} |
861 |
If the service detects an exceptional condition, it must return an error |
862 |
document with an appropriate HTTP-status code. TAP distinguishes three classes |
863 |
of exceptions. |
864 |
|
865 |
Errors in the use of the HTTP protocol. |
866 |
|
867 |
Errors in the use of the TAP protocol, including both invalid requests and |
868 |
failure of the service to complete valid requests. |
869 |
|
870 |
Error documents for HTTP-level errors are not specified in the TAP protocol. |
871 |
Responses to these errors are typically generated by service containers and |
872 |
cannot be controlled by TAP implementations. There are several cases where a |
873 |
TAP |
874 |
service could return an HTTP error. First, the /async endpoint could return a |
875 |
404 (not found) error if the client accesses a job within the UWS joblist that |
876 |
does not exist. Second, access to a resource could result in an HTTP 401 (not |
877 |
authorized) error if authentication is required or an HTTP 403 (forbidden) |
878 |
error if the client is not allowed to access the resource. |
879 |
|
880 |
Error documents for TAP errors must be VOTable documents; any result-format |
881 |
specified in the request is ignored. If the error document is being retrieved |
882 |
from the /async/<jobid>/error resource (specified by UWS) after an asynchronous |
883 |
query, the HTTP status code should be 200. If the error document is being |
884 |
returned directly after a synchronous query, the service may use an appropriate |
885 |
HTTP status code, including 200 (successfully returning a response to the |
886 |
request) and various 4xx and 5xx values. The exception condition must be |
887 |
described to the client using a status code in the VOTable header. Section |
888 |
2.9 specifies the use of VOTable for error documents in TAP services. |
889 |
|
890 |
\subsection{Overflows} |
891 |
If a query is executed by a TAP service, the number of rows in the table of |
892 |
results may exceed a limit requested by the user (using the MAXREC parameter) |
893 |
or a limit set by the service implementation (the default or maximum value of |
894 |
MAXREC). In these cases, the query is said to have 'overflowed'. Typically, a |
895 |
TAP service will not detect an overflow until some part of the table of results |
896 |
has been sent to the client. |
897 |
|
898 |
If an overflow occurs, the TAP service must produce a table of results that is |
899 |
valid, in the required output format, and which contains all the results up to |
900 |
the point of overflow. Since an output overflow is not an error condition, the |
901 |
MIME type of the output must be the same as for any successful query and the |
902 |
HTTP status-code must be as for a successful, complete query. |
903 |
|
904 |
If the output format is VOTable, section 2.9.1 describes the method by which |
905 |
the overflow is reported. No method of reporting an overflow is defined for |
906 |
formats other than VOTable. |
907 |
|
908 |
\subsection{VOTable vs RDBMS Tables and Columns} |
909 |
|
910 |
TODO: describe the bi-directional mapping of VOTable <-> RDBMS table aka the |
911 |
old section on uploading tables... should this come earlier in the doc? |
912 |
|
913 |
The column names in the transient database table are taken directly from the |
914 |
name attribute of the VOTable FIELD and PARAM elements. TODO: add column name |
915 |
restrictions here to avoid quoted identifiers. The datatypes of the |
916 |
transient table are determined from the FIELD and PARAM attributes as follows: |
917 |
|
918 |
TODO: something like the table from TAP-1.0 |
919 |
|
920 |
The default mapping of data types are shown above (no arraysize or xtype). If |
921 |
the xtype attribute is set, this is the preferred internal datatype. If xtype is |
922 |
not set, then the datatype and arraysize indicate the most suitable internal |
923 |
datatype. Note that the last column of Table (x) is not normative. |
924 |
Implementations SHOULD try to make sure that the actual types chosen are at |
925 |
least signature-compatible with the recommended types (i.e., integers should |
926 |
remain integers, floating-point values floating-point values, etc.), such that |
927 |
clients can reliably write queries against uploaded tables. |
928 |
|
929 |
For columns with xtype adql:REGION, this is particularly critical, since |
930 |
databases typically use different types to represent various STC-S objects. |
931 |
Clients are advised to assume that such columns will be approximated with |
932 |
polygons in the actual database table. |
933 |
|
934 |
In the arraysize column above, [1] means the arraysize is not set or is set to |
935 |
1, n means arraysize is set to a specific value, * means arraysize=”*”, and n* |
936 |
means arraysize=”n*” (variable size up to length n). A blank means the arraysize |
937 |
is not set. |
938 |
|
939 |
Binary values (unsignedByte in VOTable, BINARY, VARBINARY, or BLOB in ADQL) can |
940 |
be expressed as specified by the VOTable standard. By default, VOTable allows |
941 |
them to be written as an array of decimal numbers, e.g. 12 56 0 255 0 0 255 (one |
942 |
number per byte value). |
943 |
|
944 |
For columns of type BLOB or CLOB, most database systems support reference to |
945 |
these columns in the select clause but not in any other part of the query. |
946 |
Services may use these types to indicate that columns may only be selected. For |
947 |
example, if service implementors want to make URL(s) available as column values |
948 |
in the results, but do not actually store the URL(s) in the database, they would |
949 |
specify a column with xtype=”adql:CLOB” and the column with URL(s) could be |
950 |
referenced in the SELECT clause of a query, but could not be used in the WHERE |
951 |
clause. The service could then process the query result and insert the URL(s) |
952 |
or, more likely, transform a column value (an identifier) into a URL while |
953 |
writing the results. |
954 |
|
955 |
TIMESTAMP values are specified as described in [std:DALI]. The |
956 |
xtype=”adql:TIMESTAMP” attribute must be specified in an uploaded VOTable in |
957 |
order for the values to be inserted in a column of type TIMESTAMP; without the |
958 |
xtype, the values would be inserted into a CHAR(n) or VARCHAR column. |
959 |
|
960 |
POINT and REGION values are specified in STC-S format (see section 6 ). The |
961 |
xtype=”adql:POINT” attribute must be specified in an uploaded VOTable in order |
962 |
for the char values to be parsed and treated as POINTs (e.g. to be used with |
963 |
some of the ADQL region functions). For regions, the xtype=”adql:REGION” |
964 |
attribute must be specified in an uploaded VOTable in order for the char values |
965 |
to be parsed and treated as REGIONs (e.g. to be used with some of the ADQL |
966 |
region functions). |
967 |
|
968 |
\section{Metadata: TAP SCHEMA} |
969 |
|
970 |
There are several approaches to getting metadata for a given TAP service. All |
971 |
TAP services must support a set of tables in a schema named |
972 |
TAP\underline{' '}SCHEMA that describe the tables and columns included in the |
973 |
service. In addition to the TAP\underline{' '}SCHEMA, there are two other ways |
974 |
to get metadata from a TAP service. First, the VOSI tables resource provides |
975 |
metadata on all tables and columns; this resource is described in 2.2.5 . The |
976 |
VOSI tables resource provides the same metadata as the TAP\underline{' '}SCHEMA |
977 |
but in a rigorously controlled format; the information in the |
978 |
TAP\underline{' '}SCHEMA is equivalent to that defined by the VODataService |
979 |
[7]. Second, the client may specify a query of one or more tables setting the |
980 |
MAXREC parameter to 0 so that only the metadata regarding the requested fields |
981 |
is returned. Use of MAXREC is described in 2.3.7 . |
982 |
|
983 |
The TAP\underline{' '}SCHEMA provides access to table, column, and join key |
984 |
metadata through the TAP query mechanisms themselves. Users can discover tables |
985 |
or columns that meet their specific criteria by querying the tables described |
986 |
below. The service may enhance the TAP\underline{' '}SCHEMA with additional |
987 |
metadata where that seems appropriate; since it is self-describing, the |
988 |
TAP\underline{' '}SCHEMA may be queried to determine if any extended schema |
989 |
metadata is defined by the service. Services must provide these tables and make |
990 |
them accessible by all supported query mechanisms. |
991 |
|
992 |
The qualified names in the tables of the TAP schema must follow the rules |
993 |
defined in section 2.4. The names must be stated in a form that is acceptable as |
994 |
an operand of a query. |
995 |
|
996 |
All columns in the TAP\underline{' '}SCHEMA tables are of type VARCHAR except |
997 |
for size, principal, indexed, and std (in Columns) which are INTEGER values. |
998 |
|
999 |
Implementors are permitted to include additional tables in the |
1000 |
TAP\underline{' '}SCHEMA to describe additional aspects of their service not |
1001 |
covered by this specification. Implementors may also include additional columns |
1002 |
in the standard tables described below. For example, one could include a column |
1003 |
with a timestamp saying when metadata values were was last modified. |
1004 |
|
1005 |
\subsection{Schemas} |
1006 |
The table TAP\underline{' '}SCHEMA.schemas must contain the following columns: |
1007 |
|
1008 |
TODO: table |
1009 |
|
1010 |
The schema\underline{' '}name values must be unique and may be qualified by the |
1011 |
catalog name or not depending on the implementation requirements. The fully |
1012 |
qualified schema name is defined by the ADQL language and follows the pattern |
1013 |
[catalog.]schema. The schema metadata are included for reference and are not |
1014 |
used directly to construct queries. |
1015 |
|
1016 |
\subsection{Tables} |
1017 |
The table TAP\underline{' '}SCHEMA.tables must contain the following columns: |
1018 |
|
1019 |
TODO: table |
1020 |
|
1021 |
The table\underline{' '}name values must be unique. The value of the |
1022 |
table\underline{' '}name should be the string that is recommended for use in |
1023 |
querying the table; it may or may not be qualified by schema and catalog name(s) |
1024 |
depending on the implementation requirements. The fully qualified table name is |
1025 |
defined by the ADQL language and follows the pattern [[catalog.]schema.]table. |
1026 |
|
1027 |
\subsection{Columns} |
1028 |
The table TAP\underline{' '}SCHEMA.columns must contain the following columns: |
1029 |
|
1030 |
TODO: table |
1031 |
|
1032 |
The table\underline{' '}name,column\underline{' '}name (pair) values must be |
1033 |
unique. |
1034 |
|
1035 |
Data types and how they map to VOTable datatypes are described in section 2.5 |
1036 |
above. The “size” gives the length of variable length datatypes, for example |
1037 |
varchar(256); this size does not map to the VOTable arraysize attribute when the |
1038 |
latter specifies the size and shape of a multi-dimensional array. To use size in |
1039 |
a query, it must be put in double quotes since it collides with an ADQL reserved |
1040 |
word. Since delimited identifiers are case-sensitive, for the size column both |
1041 |
clients and servers MUST always (in particular, in the DDL for |
1042 |
TAP\underline{' '}SCHEMA) use lower case exclusively. In the next major version |
1043 |
of TAP, this column will be called arraysize. |
1044 |
|
1045 |
The “principal” flag indicates that the column is considered a core part the |
1046 |
content; clients can use this hint to make the principal column(s) visible, for |
1047 |
example by selecting them by default in generating an ADQL query. In cases where |
1048 |
the services selects the columns to return (such as PQL without a SELECT |
1049 |
parameter), the principal column indicates those columns that are returned by |
1050 |
default. |
1051 |
|
1052 |
The “indexed” flag indicates that the column is indexed, potentially |
1053 |
making queries run much faster if this column is used in a constraint. |
1054 |
|
1055 |
The “std” is included for compatibility with the registry, which uses this value |
1056 |
to indicate that a given column is defined by some standard, as opposed to a |
1057 |
custom column defined by a particular service. |
1058 |
|
1059 |
\subsection{Foreign Keys} |
1060 |
The table TAP\underline{' '}SCHEMA.keys must contain the following columns to |
1061 |
describe foreign key relations between tables: |
1062 |
|
1063 |
TODO: table |
1064 |
|
1065 |
The key\underline{' '}id values are unique and used only to join with the |
1066 |
TAP\underline{' '}SCHEMA.key\underline{' '}columns table below. There may be |
1067 |
one or more rows with different key\underline{' '}id values and a pair |
1068 |
of tables to denote one or more ways to join the tables. |
1069 |
|
1070 |
The table TAP\underline{' '}SCHEMA.key\underline{' '}columns must contain the |
1071 |
following columns to describe the columns that make up a foreign key: |
1072 |
|
1073 |
TODO: table |
1074 |
|
1075 |
There may be one or more rows with a specific key\underline{' '}id to |
1076 |
denote single or multi-column keys. |
1077 |
|
1078 |
A TAP service must provide the tables listed above and may provide other tables |
1079 |
in the TAP\underline{' '}SCHEMA namespace. |
1080 |
|
1081 |
|
1082 |
\section{Examples} |
1083 |
The UWS pattern is specified in [3] and its application to TAP in section |
1084 |
XREF. This section gives examples of the exchange of messages between a |
1085 |
TAP client and service when using UWS to run an asynchronous query. |
1086 |
|
1087 |
\subsection{Example: Asynchronous Query} |
1088 |
Consider a TAP service at http://example.com/tap. TAP mandates that the |
1089 |
asynchronous requests be directed to http://example.com/tap/async (e.g. for |
1090 |
anonymous queries). This URL points to the list of 'jobs'; i.e. the list of |
1091 |
queries currently or recently executed. |
1092 |
|
1093 |
\subsubsection{Creating a Query} |
1094 |
To create a new query, the client POSTs a request to the job list: |
1095 |
|
1096 |
\begin{verbatim} |
1097 |
HTTP POST http://example.com/tap/async |
1098 |
REQUEST=doQuery |
1099 |
LANG=ADQL |
1100 |
QUERY=SELECT TOP 100 * FROM foo |
1101 |
\end{verbatim} |
1102 |
|
1103 |
The service then creates a job and assigns that job a name and a URL based on |
1104 |
the name. Suppose that the name is 42, then the URL will be |
1105 |
http://example.com/tap/async/42 because the jobs are always children of the job |
1106 |
list. While the job is in the PENDING phase, additional parameters may be |
1107 |
specified by additional POSTs to the job resource, for example: |
1108 |
|
1109 |
\begin{verbatim} |
1110 |
HTTP POST http://example.com/tap/async/42 |
1111 |
UPLOAD=mytable,http://a.b.c/mytable.xml |
1112 |
\end{verbatim} |
1113 |
|
1114 |
After each such POST, the service issues an HTTP redirection to the job's URL, |
1115 |
where the modified state may be accessed: |
1116 |
|
1117 |
\begin{verbatim} |
1118 |
HTTP status 303 'See other' |
1119 |
Location: http://example.com/tap/async/42 |
1120 |
\end{verbatim} |
1121 |
|
1122 |
All TAP-specific parameters are stored using the paramList mechanism of UWS and |
1123 |
are included in the XML representation of the job: |
1124 |
\begin{verbatim} |
1125 |
HTTP GET http://example.com/tap/async/42 |
1126 |
\end{verbatim} |
1127 |
or directly from the parameters resource: |
1128 |
\begin{verbatim} |
1129 |
HTTP GET http://example.com/tap /async/42/parameters |
1130 |
\end{verbatim} |
1131 |
Individual parameters cannot be accessed as separate web resources. |
1132 |
|
1133 |
The UWS pattern requires the following resources to describe and control the |
1134 |
job: |
1135 |
\begin{verbatim} |
1136 |
http://example.com/tap/async/42/phase |
1137 |
http://example.com/tap/async/42/quote |
1138 |
http://example.com/tap/async/42/executionduration |
1139 |
http://example.com/tap/async/42/destruction |
1140 |
http://example.com/tap/async/42/results |
1141 |
http://example.com/tap/async/42/error |
1142 |
\end{verbatim} |
1143 |
The quote resource specifies the predicted completion time for the job (query), |
1144 |
assuming it is started immediately. In practice, it is very hard to estimate the |
1145 |
time a query will take; for TAP services it is recommended that this be set to |
1146 |
the current time plus the maximum amount of time the query will be allowed to |
1147 |
run (see termination below). |
1148 |
|
1149 |
The termination resource specifies the amount of time (in seconds) the job |
1150 |
(query) will be allowed to run before being aborted by the service. The |
1151 |
termination time is set by the service and can be read from the job or directly |
1152 |
from the termination resource: |
1153 |
|
1154 |
\begin{verbatim} |
1155 |
HTTP GET http://example.com/tap/async/42/executionduration |
1156 |
\end{verbatim} |
1157 |
The service may allow the client to change the termination: |
1158 |
\begin{verbatim} |
1159 |
HTTP POST http://example.com/tap/async/42/executionduration |
1160 |
TERMINATION=600 |
1161 |
\end{verbatim} |
1162 |
The destruction resource specifies when the service will destroy the job. The |
1163 |
service is only required to keep a job for a finite period of time, after which |
1164 |
it may destroy the job, including the result. After this time, the client will |
1165 |
receive an HTTP 404 'not found' status if it tries to get any information about |
1166 |
the job. The destruction time of the job is chosen by the service and the client |
1167 |
can read it from the job or directly from the destruction resource: |
1168 |
\begin{verbatim} |
1169 |
HTTP GET http://example.com/tap/async/42/destruction |
1170 |
\end{verbatim} |
1171 |
The service may allow the client to change the destruction time: |
1172 |
\begin{verbatim} |
1173 |
HTTP POST http://example.com/tap/async/42/destruction |
1174 |
DESTRUCTION=2008-11-11T11:11:11Z |
1175 |
\end{verbatim} |
1176 |
|
1177 |
\subsubsection{Running a Query} |
1178 |
The phase URL shows the progress of the job. When the job is created by the |
1179 |
service it will normally be set to PENDING, but might be set to ERROR if the |
1180 |
service has rejected the job. If the phase is ERROR, then the error URL should |
1181 |
lead to a an error document explaining the problem. If the phase is PENDING, |
1182 |
then the client needs to commit the job for execution. |
1183 |
|
1184 |
The client runs the job by posting to the phase URL: |
1185 |
\begin{verbatim} |
1186 |
HTTP POST http://example.com/tap /async/42/phase |
1187 |
PHASE=RUN |
1188 |
\end{verbatim} |
1189 |
|
1190 |
The service replies with a redirection to the job URL |
1191 |
\begin{verbatim} |
1192 |
HTTP status 303 'see other' |
1193 |
Location: http://example.com/tap /async/42 |
1194 |
\end{verbatim} |
1195 |
The phase will now have changed to either QUEUED or EXECUTING, depending on the |
1196 |
service implementation. The client tracks the execution by polling the phase |
1197 |
URL: |
1198 |
\begin{verbatim} |
1199 |
HTTP GET http://example.com/tap/async/42/phase |
1200 |
\end{verbatim} |
1201 |
A job in the QUEUED or EXECUTING phase may be aborted by posting to the phase |
1202 |
URL: |
1203 |
\begin{verbatim} |
1204 |
HTTP POST http://example.com/tap/async/42/phase |
1205 |
PHASE=ABORT |
1206 |
\end{verbatim} |
1207 |
|
1208 |
When the query is complete, the phase changes to COMPLETED. The client then |
1209 |
retrieves the result from the results list: |
1210 |
\begin{verbatim} |
1211 |
HTTP GET http://example.com/tap/async/42/results/result |
1212 |
\end{verbatim} |
1213 |
The client knows that the table of results is at the URL /result relative to the |
1214 |
results list because the TAP protocol requires this naming. A generic UWS client |
1215 |
could find the name of the result and retrieve it by examining either the job |
1216 |
description: |
1217 |
\begin{verbatim} |
1218 |
HTTP GET http://example.com/tap/async/42 |
1219 |
\end{verbatim} |
1220 |
or by looking specifically at the result list: |
1221 |
\begin{verbatim} |
1222 |
HTTP GET http://example.com/tap/async/42/results |
1223 |
\end{verbatim} |
1224 |
If the service cannot run the query, then the final phase is ERROR and there is |
1225 |
no table of results. In this case, the client should expect an HTTP 404 'not |
1226 |
found' status if it tries to retrieve the result. The client should look instead |
1227 |
at the error resource to find out what went wrong: |
1228 |
\begin{verbatim} |
1229 |
HTTP GET http://example.com/tap/async/42/error |
1230 |
\end{verbatim} |
1231 |
If the job was aborted (by the client or the service), the final phase will be |
1232 |
ABORTED and there is no table or results. As with errors, the client should look |
1233 |
at the error resource to find out what went wrong. |
1234 |
|
1235 |
The basic sequence can be executed from a web browser or from a shell script |
1236 |
using the curl utility: |
1237 |
\begin{verbatim} |
1238 |
curl -d 'REQUEST=doQuery&LANG=PQL&POS=12,34&SIZE=0.5&FROM=foo' \ |
1239 |
http://example.com/tap/async |
1240 |
|
1241 |
[read Location header from curl output] |
1242 |
|
1243 |
curl -d 'PHASE=RUN' http://example.com/tap/async/42 |
1244 |
|
1245 |
curl http://example.com/tap/async/42/phase |
1246 |
|
1247 |
[repeat until phase is COMPLETED] |
1248 |
|
1249 |
curl http://example.com/tap/42/results/result |
1250 |
\end{verbatim} |
1251 |
|
1252 |
\subsubsection{Example: Synchronous Query} |
1253 |
|
1254 |
TODO |
1255 |
|
1256 |
\subsubsection{Example: DALI-examples Document} |
1257 |
|
1258 |
TODO |
1259 |
|
1260 |
\section{Use of STC-S in TAP - deprecated} |
1261 |
|
1262 |
NOTE: The old section 6 is not included from TAP-1.0 since it was informative |
1263 |
and does not belong in this document. If we need to define syntax for |
1264 |
points, circles, and polygons then that should be done in [std:DALI], which |
1265 |
already defines timestamp syntax. |
1266 |
|
1267 |
\section{VOSpace Integration - deprecated} |
1268 |
|
1269 |
Note: some text moved to the UPLOAD section; may propose DEST parameter for |
1270 |
outout. |
1271 |
|
1272 |
\section{Use of HTTP - deprecated} |
1273 |
Note: This section is in or belongs in DALI. |
1274 |
|
1275 |
\appendix |
1276 |
|
1277 |
\section{Changes from Previous Versions} |
1278 |
|
1279 |
\subsection{Changes from TAP-1.0} |
1280 |
|
1281 |
Restructured the document and removed text that duplicates material from DALI. |
1282 |
Rewrite the overly long introduction with some basic use cases to help define |
1283 |
the scope and tell readers what TAP is supposed to accomplish. |
1284 |
|
1285 |
Made clarifications: restricted allowed table names for UPLOAD, clarified that |
1286 |
multiple UPLOAD pamaters accumulate, deprecated the size column in |
1287 |
TAP\underline{' '}SCHEMA.columns and added advice to quote it as a delimited |
1288 |
identifier, made presence of a TABLE element on VOTable output only required for |
1289 |
successful queries, added optional DALI-examples endpoint (text TBD). |
1290 |
|
1291 |
Defined standardID values for the async and sync resource types and explicitly |
1292 |
allow for multiple of each resource (typically to support authentication). The |
1293 |
fixed paths /async and /sync are still required and are to provide anonymous |
1294 |
query access, which should be compatible with existing services. |
1295 |
|
1296 |
\bibliography{ivoatex/ivoabib} |
1297 |
|
1298 |
|
1299 |
\end{document} |