/[volute]/branches/SODA-markus/SODA.tex
ViewVC logotype

Contents of /branches/SODA-markus/SODA.tex

Parent Directory Parent Directory | Revision Log Revision Log


Revision 3230 - (show annotations)
Tue Jan 26 12:32:11 2016 UTC (5 years, 8 months ago) by msdemlei
File MIME type: application/x-tex
File size: 35604 byte(s)
SODA-branch: adding an explanatory supplement.


1 \documentclass[11pt,a4paper]{ivoa}
2 \input tthdefs
3
4 \usepackage{listings}
5 \lstloadlanguages{XML,sh}
6 \lstset{flexiblecolumns=true,tagstyle=\ttfamily,
7 showstringspaces=False}
8 \usepackage{todonotes}
9
10 \title{IVOA Server-side Operations for Data Access}
11
12 \ivoagroup{DAL}
13
14 \author{Fran\c cois Bonnarel, Markus Demleitner, Patrick Dowler, Douglas Tody }
15
16 \editor{Fran\c cois Bonnarel}
17
18 \previousversion{WD-AccessData-1.0-20151021}
19 \previousversion{WD-AccessData-1.0-20140730}
20 \previousversion{WD-AccessData-1.0-20140312}
21
22
23 \begin{document}
24
25 \begin{abstract}
26 This document describes the SODA web service capability. SODA is a low-level data access capability or server side data processing that can act upon the data files, performing various kinds of operations: filtering/subsection, transformations, pixel operations, and applying functions to the data.
27
28 \end{abstract}
29
30 \section*{Acknowledgments}
31 The authors would like to thank all the participants in DAL-
32 WG discussions for their ideas, critical reviews, and
33 contributions to this document.
34
35 \section{Introduction}
36 The SODA web service interface defines a RESTful web service for performing server-side operations and processing on data before transfer.
37
38
39 \subsection{The Role in the IVOA Architecture}
40
41
42
43
44 TODO: new diagram from TCG
45
46 SODA services conform to the Data Access
47 layer Interface \citep{std:DALI} specification, including the
48 Virtual Observatory Support Interfaces \citep{std:VOSI} resources.
49
50 \subsection{Motivating Use Cases}
51 Below are some of the more common use cases that have motivated the development of the SODA specification. While this is not complete, it helps to understand the problem area covered by this specification.
52
53 \subsubsection{Retrieve Subsection of a Datacube}
54 \label{sect:use-cube}
55
56 Cutout a subsection using coordinate axis values. The input to the cutout operation will include one or more of the following:
57
58
59 \begin{itemize}
60 \item a region on the sky
61 \item an energy value or range
62 \item a time value or range
63 \item one or more polarization states
64 \end{itemize}
65
66 The region on the sky should be something simple: a circle,
67 a range of coordinate values, or maybe a polygon.
68
69 \subsubsection{Retrieve subsection of a 2D Image}
70 This is a special case of \ref{sect:use-cube},
71 where the cutout is only in the spatial axes.
72
73 \subsubsection{Retrieve subsection of a Spectrum}
74
75 This is a special case of \ref{sect:use-cube},
76 where the cutout is only in the spectral axis.
77
78 \subsection{Provide the data in different formats}
79
80 Examples are images in PNG, or JPEG instead of FITS and spectra in csv, FITS or VOTable.
81
82 \subsubsection{Flatten a Datacube into a 2D Image}
83
84 This use case will be developed and supported in the
85 SODA-1.1 (or later) specification.
86
87 \subsubsection{Flatten a Datacube into a 1D Spectrum}
88
89 This use case will be developed and supported in the
90 SODA-1.1 (or later) specification.
91
92 \subsubsection{Rebin Data by a Fixed Factor}
93
94 This use case will be developed and supported in the
95 SODA-1.1 (or later) specification.
96
97 \subsubsection{Reproject Data onto a Specified Grid}
98
99 This use case will be developed and supported in the
100 SODA-1.1 (or later) specification.
101
102 \subsubsection{Compute Aggregate Functions over the Data}
103
104 This use case will be developed and supported in the
105 SODA-1.1 (or later) specification.
106
107
108 \subsubsection{Apply Standard Function to Data Values}
109
110 It could be
111 ``denoising'' with standard methods or ``on the fly'' recalibration. This use case will be developed and supported in the
112 SODA-1.1 (or later) specification.
113
114 \subsubsection{Apply Arbitrary User-Specified Function to Data Values}
115
116 This use case will be developed and supported in the
117 SODA-1.1 (or later) specification.
118
119 \subsubsection{Run Arbitrary User-Supplied Code on the Data}
120
121 This use case will be developed and supported in the
122 SODA-1.1 (or later) specification.
123
124 \subsection{SODA Operation}
125
126 In contrast to other IVOA protocols, SODA services are not usually
127 discovered through Registry queries. Instead, clients encounter them
128 in Datalink \citep{std:datalink} declarations, which can either be
129 standalone or embedded within other services' responses.
130
131 Since this pattern can appear somewhat confusing at first, this
132 introductory (non-normative) chapter discusses the scenarios in which
133 clients can encounter SODA services. In parallel, we provide advice on
134 the server-side implications of these scenarios.
135
136 In all cases, the first step is a data discovery service; when used
137 below, this term could refer to, for instance, SIA, SSA, or ObsTAP, but also
138 to some sort of resolution engine for persistent identifiers.
139
140 \subsubsection{Pure Datalink discovery}
141 \label{sect:pure-datalink}
142
143 In this scenario, in the data discovery step, the service has returned a
144 row that declared a result with the media type
145 $$\hbox{\texttt{application/x-votable+xml;content=datalink}}.$$
146
147 To the client, this indicates that what is given in the access reference
148 (e.g., the access\_url column in ObsTAP or SIA version 2 or the column
149 with the UCD VOX:Image\_AccessReference in SIA version 1) is a datalink
150 document. Within that document, there is a datalink service descriptor
151 with certain properties defined in this document that looks somewhat
152 like this:\todo{of course,
153 this needs descriptions and ranges; if this text is accepted for the
154 main standard, MD will fill this in.}
155
156 \begin{lstlisting}[language=XML,basicstyle=\footnotesize]
157 <VOTABLE...>
158 <RESOURCE type="results">
159 [datalink links, one of them being]
160 <TR>[id=ivo://example.com/data?ds1 service-def=soda; semantics=#proc]</TR>
161 </RESOURCE>
162
163 <RESOURCE type="meta" utype="adhoc:service" ID="soda">
164
165 <PARAM name="standardID" datatype="char" arraysize="*"
166 value="ivo://ivoa.net/std/SODA#sync-1.0" />
167
168 <PARAM name="accessURL" datatype="char" arraysize="*"
169 value="http://example.com/my-svcs/soda/sync?ID=ivo://example.com/data?ds1" />
170
171 <GROUP name="inputParams">
172 <PARAM name="POS" ucd="phys.angArea;obs" datatype="char"
173 arraysize="*" />
174 <PARAM name="BAND" ucd="em.wl" unit="m" datatype="double"
175 arraysize="*"/>
176 <PARAM name="TIME" ucd="time.interval;obs.exposure"
177 unit="d" datatype="double"
178 arraysize="*" xtype="interval" />
179 <PARAM name="POL" ucd="meta.code;phys.polarization" datatype="char"
180 arraysize="*" />
181 </GROUP>
182 </RESOURCE>
183 \end{lstlisting}
184
185 Of course, the service is free to choose the VOTable ID of the
186 adhoc:service resource; the service will only declare the parameters it
187 (and the underlying data) actually supports.
188
189 From the datalink line, the client sees that there is a service for the
190 dataset in question (identified here through its publisher DID,
191 \nolinkurl{ivo://example.com/data?ds1}), and from the service
192 descriptor's standardID \xmlel{PARAM} it learns that the service's
193 parameters follow the rules laid down here, in particular as regards the
194 three-factor semantics. For instance, the client is guaranteed that
195 BAND, with UCD em.wl and unit meters actually denotes the parameter
196 controlling where a cutout on the spectral axis will happen.
197
198 SODA's role here is exactly this guarantee of a specific semantics, as
199 opposed to a non-standard service that could use BAND in an entirely
200 different way.
201
202 An attractive implementation strategy for small-to-medium sized
203 installations is to pre-generate the datalink files. In that way, no
204 extra endpoint is required besides the discovery service and the SODA
205 service.
206
207 Here is a sketch of the query pattern in this case\todo{If people think
208 this is a good idea, I'll do SVGs of these}:
209
210 \begin{verbatim}
211 Client ---- discovery query ----> DAL service
212 |
213 +----- Results with ------------+
214 | Datalink-valued accrefs
215 v
216 Datalink client --- retrieved accref ---> e.g., plain HTTP
217 service
218 |
219 +-------- Datalink document with -----------+
220 | SODA descriptor
221 v
222 SODA client -----> SODA instructions ----> SODA service
223 |
224 Data viewer <------ sliced-and-diced data -----+
225 \end{verbatim}
226
227 \subsubsection{Datalink Discovery with Backward Compatiblity}
228 \label{sect:dlplusbackward}
229
230 The problem with the scheme discussed in sect.~\ref{sect:pure-datalink} is
231 that legacy clients, i.e., those that do not understand Datalink, will
232 not be able to interpret the results of the discovery step. While this
233 is probably desirable when services hand out large data cubes that
234 legacy clients probably will not properly handle anyway, in many other
235 situations it is desirable to deliver conventional (e.g., FITS) data
236 products to such legacy clients. To still enable SODA and other
237 datalink functionality, DAL services can add a serivce descriptor in the
238 DAL response that indicates the availability of a Datalink
239 \emph{service} accompanying the DAL service, looking more or less like
240 this:
241
242 \begin{lstlisting}[language=XML]
243 <RESOURCE type="results">
244 [a result from services like TAP, SIA, SSA]
245 <TABLE>
246 [in particular, we have one field like]
247 <FIELD ID="primaryID" name="pubDID" datatype="char" arraysize="*">
248 <DESCRIPTION>The publisher DID for the dataset</DESCRIPTION>
249 </FIELD>
250 ...
251 </TABLE>
252 </RESOURCE>
253 <RESOURCE type="meta" utype="adhoc:service">
254 <PARAM name="standardID" datatype="char" arraysize="*"
255 value="ivo://ivoa.net/std/DataLink#links-1.0" />
256 <PARAM name="accessURL" datatype="char" arraysize="*"
257 value="http://example.com/mylinks/get" />
258 <GROUP name="inputParams">
259 <PARAM name="ID" datatype="char" arraysize="*"
260 value="" ref="primaryID"/>
261 </GROUP>
262 </RESOURCE>
263 \end{lstlisting}
264
265 Note that while this looks very similar to the SODA descriptor above,
266 this fragment is in the DAL response and references one (or more)
267 field(s) from the DAL response, whereas the SODA descriptor resides in a
268 Datalink document. This is explained in more detail in section 4.2 of
269 the Datalink recommendation 1.0. The net result is that
270 datalink-enabled clients can find ancillary data and use SODA services
271 for data access by virtue of being able to retrieve Datalink documents,
272 whereas legacy clients still retain basic functionality.
273
274 On the service side, this incurs the additional cost of having to
275 provide a datalink \{links\} resource, on the client side, some extra
276 dereferencing becomes necessary. Hence, this pattern should be
277 preferred over the simpler pattern from sect.~\ref{sect:pure-datalink}
278 only if there is a significant advantage in serving data to legacy
279 clients.
280
281 The query pattern in this case looks like this:
282
283 \begin{verbatim}
284
285 Client ---- discovery query ----> DAL service
286 |
287 +----- Results with ------------+
288 | pubDIDs and a {links} descriptor
289 v
290 Datalink client ----- ID=pubDID -----> Datalink service
291 |
292 +-------- datalink document with -----------+
293 | SODA descriptor
294 v
295 SODA client -----> SODA instructions ----> SODA service
296 |
297 Data viewer <------ sliced-and-diced data -----+
298 \end{verbatim}
299
300 \subsubsection{Sidestepping Datalink}
301
302 In some situations, the extra request to retrieve the datalink document
303 for each dataset is inconvenient, while the client may have sufficient
304 information to operate the SODA service based on common metadata. A
305 classic example would be a service containing relatively homogeneous
306 results of a single instrument, perhaps a spectrograph where all
307 spectra essentially have the same spectral coverage and a client may
308 want to only retrieve, say, the vicinity of a spectral line.
309
310 In such cases a service may provide a shortcut by including a SODA
311 descriptor directly in the DAL response. In essence the resulting
312 descriptor looks like a union of the one given in
313 sect.~\ref{sect:pure-datalink} and the one given in
314 sect.~\ref{sect:dlplusbackward}: It includes the SODA parameters, the ID
315 parameter with the reference to the column to take the publisher DID
316 from, but it has a SODA standardID from sect.~\ref{sect:pure-datalink}
317 rather than the Datalink one from sect.~\ref{sect:dlplusbackward}.
318
319 While sidestepping the extra datalink request might appear attractive in
320 principle, the difficulty of determining the useful parameter ranges
321 make this pattern only interesting in relatively few special cases.
322 Clients must not rely on the presence of full SODA descriptors in DAL
323 responses. Normal SODA operation follows the pattern given in
324 sects.~\ref{sect:pure-datalink} and~\ref{sect:dlplusbackward}.
325
326 The query pattern here is:
327
328 \begin{verbatim}
329 Client ---- discovery query ----> DAL service
330 |
331 +----- Results with ------------+
332 | SODA descriptor
333 v
334 SODA client -----> SODA instructions ----> SODA service
335 |
336 Data viewer <------ sliced-and-diced data -----+
337 \end{verbatim}
338
339
340 \section{Resources}
341
342 SODA services are implemented as HTTP REST \citep{richardson07} web
343 services with a \{sync\} resource that conforms to the DALI-
344 sync resource description.
345
346 \begin{table}[ht]
347 \begin{tabular}{rrr}
348 \sptablerule
349 \textbf{resource type}&\textbf{resource name}&\textbf{required}\cr
350 \sptablerule
351 \{sync\}&service specific&\cr
352 \{async\}&service specific&\cr
353 {DALI-examples}&/examples&no\cr
354 {VOSI-availability}&/availability&yes\cr
355 {VOSI-capabilities}&/capabilities&yes\cr
356 \sptablerule
357 \end{tabular}
358 \caption{Endpoints for AccessData services}
359 \end{table}
360
361 A stand-alone SODA service may have one or both of the \{sync\} and \{async\} resources. For either type, it could have multiple resources (e.g. to support alternate authentication schemes). The SODA service may also include other custom or supporting resources.
362
363 Either the \{sync\} or \{async\} SODA capability may be included as part of other web services. For example, a single web service could contain the SIA-2.0 \{query\} capability, the DataLink-1.0 \{links\} capability, and the SODA \{sync\} capability. Such a service must also have the VOSI-availability and VOSI-capabilities resources to report on and describe all the implemented capabilities.
364
365 \subsection{\{sync\} resource}
366 \label{sect:syncres}
367
368 The \{sync\} resource is a synchronous web service resource
369 that conforms to the DALI-sync description. Implementors
370 are free to name this resource however
371 they like, except that the name must consist of one URI segment only (i.e.,
372 contain no slash). This is to allow clients, given the access URL,
373 can reliably find out the URL of the capabilities endpoint.
374 Clients, in turn, can find the resource path using the
375 VOSI-capabilities resource, but will in general be provided the access
376 URLs through a previous data discovery query or through direct user
377 input.
378
379 The \{sync\} resource performs the data access as specified by
380 the input parameters and returns the data directly in the
381 output stream. Synchronous data access is suitable when the
382 operations can be quickly performed and the data stream can
383 be setup and written to (by the service) in a short period
384 of time (e.g. before any timeouts).
385
386 \subsection{\{async\} resource}
387
388 The \{async\} resource is an asynchronous web service resource
389 that conforms to the DALI-async description. The considerations on
390 naming the resource given in sect.~\ref{sect:syncres} apply for it.
391
392 The \{async\} resource performs the data access as specified
393 by the input parameters and either (i) stores the results
394 for later transfer or (ii) pushes the results to a specified
395 destination (e.g. to a VOSpace location). Asynchronous data
396 access usually introduces resource constraints on the
397 service (which may be limited) and usually imposes a higher
398 latency before any results can be seen because the location
399 of results does not have to be valid until the data access
400 job is complete. Asynchronous data access is intended for
401 (but not limited to) use when the operations take
402 considerable time and results must be staged (e.g. some
403 multi-pass algorithms or operations that result in multiple
404 outputs).
405
406 \subsection{Examples: DALI-examples}
407
408 SODA services should provide a DALI-examples resource
409 with one example invocation that shows the variety
410 operations the service can perform. Example operations using
411 the \{sync\} resource and that output a small data stream are
412 preferred, as the examples may be used by automatic validators doing
413 relatively frequent (of order daily) queries.
414
415 Parameters to be passed to the service must be given using the DALI
416 \texttt{generic-parameter} term.
417
418
419 \subsection{Availability: VOSI-availability}
420
421 A SODA web service must have a VOSI-availability
422 resource \citep{std:VOSI} as described in DALI \citep{std:DALI}.
423
424 \subsection{Capabilities: VOSI-capabilities}
425
426 A web service that includes SODA capabilities must
427 have a VOSI-capabilities resource \citep{std:VOSI} as described in DALI
428 \citep{std:DALI}. The standardID for the \{sync\} resource is
429 $$\hbox{\texttt{ivo://ivoa.net/std/SODA\#sync-1.0}.}$$
430
431 The standardID for the \{async\} resource is
432
433 $$\hbox{\texttt{ivo://ivoa.net/std/SODA\#async-1.0}.}$$
434
435 All DAL services must implement the \texttt{/capabilities} resource.
436 The following capabilities document shows the minimal
437 metadata for a stand-alone SODA service and does not
438 require a registry extension schema:
439
440 \begin{lstlisting}[language=XML]
441 <?xml version="1.0"?>
442 <capabilities
443 xmlns:vosi="http://www.ivoa.net/xml/VOSICapabilities/v1.0"
444 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
445 xmlns:vod="http://www.ivoa.net/xml/VODataService/v1.1">
446 <capability standardID="ivo://ivoa.net/std/VOSI#capabilities">
447 <interface xsi:type="vod:ParamHTTP" version="1.0">
448 <accessURL use="full">http://example.com/data/capabilities</accessURL>
449 </interface>
450 </capability>
451 <capability standardID="ivo://ivoa.net/std/VOSI#availability">
452 <interface xsi:type="vod:ParamHTTP" version="1.0">
453 <accessURL use="full">
454 http://example.com/data/availability
455 </accessURL>
456 </interface>
457 </capability>
458 <capability standardid="ivo://ivoa.net/std/SODA#sync-1.0">
459 <interface xsi:type="vod:ParamHTTP" role="std" version="1.0">
460 <accessurl use="full">
461 http://example.com/data/sync
462 </accessurl>
463 </interface>
464 <!-- service details from extension schema could go here -->
465 </capability>
466 <capability standardid="ivo://ivoa.net/std/SODA#async-1.0">
467 <interface xsi:type="vod:ParamHTTP" role="std" version="1.0">
468 <accessurl use="full">
469 http://example.com/data/async
470 </accessurl>
471 </interface>
472 <!-- service details from extension schema could go here -->
473 </capability>
474 </capabilities>
475 \end{lstlisting}
476
477 Note that the \{sync\} and \{async\} resources do not have to be
478 named as shown in the accessURL(s) above. Multiple
479 capability elements for the \{sync\} and the \{async\} resources
480 may be included; this is typically used if the differ in
481 protocol (http vs. https) and/or authentication
482 requirements.
483
484 \begin{admonition}{Note}
485 As SODA builds upon several concepts of DataLink \citep{std:datalink},
486 that document should be read before trying to understand the following
487 material.
488 \end{admonition}
489 \subsection{Parameter Description and Three-Factor Semantics}
490
491 \subsubsection{Three-factor Semantics}
492
493 Parameters in SODA are defined by triples of name, UCD, and unit (the
494 ``SODA triple''). Data services are free to support as many such parameters
495 as is appropriate for their datasets, in addition to supporting
496 standard parameters. With the three factors, it is
497 unlikely that two operators will by accident use the same three factors
498 for parameters of differing semantics. In this way, when a new standard
499 parameter is adopted, no existing service should require changes to
500 remain compliant.
501
502 With standard parameters as defined in this document, clients can rely
503 on certain semantics and exploit that knowledge in the provision of
504 special UIs or APIs. Standard parameters defined so far are given
505 in table~\ref{table:standardpars}. Instructions for how to propose
506 additional standard parameters are given on the landing page of the IVOA
507 DAL working group\footnote{At the time of writing, this is
508 \href{http://wiki.ivoa.net/twiki/bin/view/IVOA/IvoaResReg}{http://wiki.ivoa.net/twiki/bin/view/IVOA/IvoaResReg}.}.
509
510 \begin{table}[mht]
511 \begin{tabular}{llll}
512 \sptablerule
513 \textbf{Name}&\textbf{UCD}&\textbf{Unit}&\textbf{Semantics}\cr
514 \sptablerule
515 ID&meta.ref.url;meta.curation&&cf.~sect.~\ref{sect:ID}\cr
516 POS&phys.angArea;obs&&cf.~sect.~\ref{sect:POS}\cr
517 BAND&em.wl&m&cf.~sect.~\ref{sect:BAND}\cr
518 TIME&time.interval;obs.exposure&d&cf.~sect.~\ref{sect:TIME}\cr
519 POL&meta.code;phys.polarization&&cf.~sect.~\ref{sect:POL}\cr
520 \sptablerule
521 \end{tabular}
522 \caption{SODA triples for the standard parameters defined here.}
523 \label{table:standardpars}
524 \end{table}
525
526 Both standard and non-standard parameters should follow DALI conventions
527 if at all possible. Roughly, float-valued parameters should be mapped
528 to interval-valued parameters (i.e., do not split up minimum and maximum
529 into separate parameters). Depending on their semantics, integer
530 parameters should either be intervals or enumerated parameters (which
531 typically can be repeated). String-valued parameters should always
532 be enumerated.
533
534 \subsubsection{Discovery of Supported Parameter. Implementation strategies}
535 \label{sect:discovery-supported}
536
537 There are two principal ways a client will be led to a SODA service:
538 From a service descriptor that is part of a DAL discovery response,
539 or by direct access mediated or not via a DataLink service. In the latter case an empty query to the service iwill answer by giving the service descriptor. . In both cases,
540 clients must obtain the set of supported parameters
541 from the applicable service descriptor.
542
543 In contrast to previous IVOA DAL protocols, SODA is not a data discovery
544 protocol but instead operates on concrete datasets. In a typical case,
545 some combinations of parameters (e.g., a positional or spectral cutout)
546 may yield no output at all, as the coverage of an individual dataset is
547 very limited. To provide meaningful user interfaces, clients would therefore
548 need detailed information on the service parameters, in particular as
549 regards their domains (i.e., set of values that are likely to yield
550 non-empty results). This, in regard, requires efficient and rich service APIs, not unlikely involving service-specific parameters.
551
552 But the double ways towards SODA service execution makes the providing of this information difficult to describe and is not done in this version.
553
554 As a consequence of this, when the SODA service is driven from a data discovery phase, parameters domains can be inferred by the end-user from the discovery reponse. In the other case, it may happen that these parameters have to be furnished blindly by the end-user.
555
556 \subsubsection{SODA Service Descriptor}
557 \label{sect:serv-desc}
558
559 The DataLink \citep{std:datalink} specification describes a mechanism for
560 describing a service within a VOTable resource and
561 recommends that services can describe themselves with a
562 special resource with \texttt{name="this"}. SODA responses for
563 empty sync queries should include a descriptor describing
564 both standard and custom query parameters (if applicable).
565 The descriptor for a service with standard parameters (see
566 sect.~\ref{sect:stdpars}) would be:
567
568 \begin{lstlisting}[language=XML,basicstyle=\footnotesize]
569
570 <RESOURCE type="meta" utype="adhoc:service" name="this">
571
572 <PARAM name="standardID" datatype="char" arraysize="*"
573 value="ivo://ivoa.net/std/SODA#sync-1.0" />
574
575 <PARAM name="accessURL" datatype="char" arraysize="*"
576 value="http://example.com/SODA/sync" />
577
578 <GROUP name="inputParams">
579 <PARAM name="ID" ucd="meta.ref.url;meta.curation" datatype="char"
580 arraysize="*" />
581 <PARAM name="POS" ucd="phys.angArea;obs" datatype="char"
582 arraysize="*" />
583 <PARAM name="BAND" ucd="em.wl" unit="m" datatype="double"
584 arraysize="*"/>
585 <PARAM name="TIME" ucd="time.interval;obs.exposure"
586 unit="d" datatype="double"
587 arraysize="*" xtype="interval" />
588 <PARAM name="POL" ucd="meta.code;phys.polarization" datatype="char"
589 arraysize="*" />
590 </GROUP>
591 </RESOURCE>
592
593 \end{lstlisting}
594
595 This VOTable resource should be output for empty sync
596 queries; Thus all inputs and outputs would be fully
597 described.
598
599 \subsubsection{Client Handling of Discovered Parameters}
600
601 To keep SODA clients useful even when advanced data products are being
602 accessed, the following procedure should be followed by clients:
603
604 \begin{enumerate}
605 \item Obtain the parameter triples domains as
606 described in sect.~\ref{sect:discovery-supported}.
607 \item Identify standard parameters understood by the client and provide
608 suitable input widgets (e.g., a rubberband on top of a sky rendering,
609 or a \texttt{spatial\_cutout} method possibly allowing library-specific
610 coverage objects) for them.
611
612 \item For the remaining parameters, provide generic UI or API elements
613 (e.g., sliders or text boxes with domains annotated for intervals, pop-up
614 menus for enumerated values).
615 \end{enumerate}
616
617
618
619
620 \section{Parameters for \{sync\} and \{async\}}
621
622 \label{sect:stdpars}
623
624 The \{sync\} and \{async\} resources accept the same set of
625 parameters.
626
627
628 \subsection{Common Parameters}
629
630 \subsubsection{ID}
631 \label{sect:ID}
632
633 The ID parameter is used to specify the dataset or file to
634 be accessed. The values for the ID parameter are generally
635 discovered from data discovery or DataLink requests. The
636 values must be treated as opaque identifiers that are used
637 as-is. The DataLink specification \citep{std:datalink} describes mechanisms
638 for conveying opaque parameters and values in service
639 descriptor resources that can be used by clients to set the
640 ID parameter.
641
642 The ID parameter is single-valued in \{sync\} requests, so
643 \{sync\} soda requests access a single dataset or file.
644 Multiple ID parameters may be submitted in \{async\} requests
645 on order to bundle access to multiple datasets or files in a
646 single job.
647
648
649 The ID ucd is ``meta.ref.url;meta.curation'', and its unit is blank.
650 In addition its datatype ``char'', with
651 arraysize ``*''.
652
653
654 \subsection{Filtering Parameters}
655
656 Filtering parameters are used to extract subsets of larger
657 datasets or data files. In general, filtering parameters are
658 single-valued in \{sync\} requests and multi-valued in \{async\}
659 requests (exceptions noted below). When multiple values of
660 filtering parameters are used in an \{async\} job, each
661 combination of values produces zero or one result. For
662 example, if an \{async\} job included two POS and two BAND
663 values, there could be as many as four results (or fewer if
664 some combinations do not produce a result because the filter
665 does not intersect the bounds of the data).
666
667 \subsubsection{POS}
668 \label{sect:POS}
669
670 The POS parameter defines the positional region(s) to be
671 extracted from the data. The value is made up of a shape
672 keyword followed by coordinate values. The
673 allowed shapes are:
674
675 \begin{table}[h]
676 \begin{tabular}{ll}
677 \sptablerule
678 \textbf{Shape}&\textbf{Coordinate values}\cr
679 \sptablerule
680 \texttt{CIRCLE}&\texttt{<longitude> <latitude> <radius>}\cr
681 \texttt{RANGE}&\texttt{<longitude1> <longitude2> <latitude1> <latitude2>}\cr
682 \texttt{POLYGON}&\texttt{<longitude1> <latitude1> ... (at least 3 pairs)}\cr
683 \sptablerule
684 \end{tabular}
685 \caption{POS Values in Spherical Coordinates}
686 \end{table}
687
688 As in DALI, open intervals use -Inf or +Inf as one limit.
689
690 Examples for POS values:
691
692 \begin{itemize}
693 \item A circle at (12,34) with radius 0.5:
694
695 \begin{lstlisting}
696 POS=CIRCLE 12 34 0.5
697 \end{lstlisting}
698
699 \item A range of [12,14] in longitude and [34,36] in latitude:
700
701 \begin{lstlisting}
702 POS=RANGE 12 14 34 36
703 \end{lstlisting}
704
705 \item A polygon from (12,34) to (14,34) to (14,36) to (12,36) and
706 (implicitly) back to (12,34):
707
708 \begin{lstlisting}
709 POS=POLYGON 12 34 14 34 14 36 12 36
710 \end{lstlisting}
711
712 The inside is always assumed to be the smaller of the region
713 to the left and the region to the right so only polygons
714 smaller than half the sphere can be specified.
715
716 \item A band around the equator:
717
718 \begin{lstlisting}
719 POS=RANGE 0 360 -2 2
720 \end{lstlisting}
721
722 \item The north pole:
723
724 \begin{lstlisting}
725 POS=RANGE 0 360 89 +Inf
726 \end{lstlisting}
727 \end{itemize}
728
729 All longitude and latitude values (plus the radius of the
730 CIRCLE) are expressed in degrees in the ICRS. A future
731 version of this specification may allow the use of other
732 reference systems (specifically the native system of the
733 data).
734
735 The POS parameter is single-valued for \{sync\} requests and
736 multi-valued for \{async\} jobs.
737
738 Since it is string-valued, POS is unitless (although the values
739 contained in the the string are all in decimal degrees).
740 The ucd is ``phys.angArea;obs''. However the datatype
741 of the POS parameter is ``char''.
742
743 \subsubsection{BAND}
744 \label{sect:BAND}
745
746 The BAND parameter defines the energy interval(s) to be
747 extracted from the data. The value is an open or closed
748 numeric interval of values in the native spectral axis
749 coordinate system and units of the data. The intervals
750 always include the bounding values. As in DALI, open intervals use -Inf
751 or +Inf as one limit.
752
753 If there is one single value the interval is assumed to be
754 infinitely small (a scalar value).
755
756 \begin{itemize}
757 \item The closed interval [500,550]:
758
759 \begin{lstlisting}
760 BAND=500 550
761 \end{lstlisting}
762
763 \item The open interval (-inf,300]:
764
765 \begin{lstlisting}
766 BAND=-Inf 300
767 \end{lstlisting}
768
769 \item The open interval [750,inf):
770
771 \begin{lstlisting}
772 BAND=750 +Inf
773 \end{lstlisting}
774
775 \item The scalar value 550, equivalent to [550,550]:
776
777 \begin{lstlisting}
778 BAND=550
779 \end{lstlisting}
780
781 \end{itemize}
782
783 Extracting using a scalar value should normally extract a
784 single pixel along the energy axis of the data; extracting
785 using an interval should extract one or more pixels.
786
787 All energy values are expressed as barycentric wavelength in
788 meters. A future version of this specification may allow the
789 use of other reference systems (specifically the native
790 system of ther data).
791
792 The BAND parameter is single-valued for \{sync\} requests and
793 multi-valued for \{async\} jobs.
794
795 The ucd of the BAND parameter is ``em.wl'', the unit is ``m''. Its datatype is
796 double with an arraysize of 2, and the xtype is ``interval'' as defined in DALI.
797
798
799 \subsubsection{TIME}
800 \label{sect:TIME}
801
802 The TIME parameter defines the time interval(s) to be
803 extracted from the data. The value is an open or closed
804 interval with either numeric values (interpreted as Modified
805 Julian Dates). As in DALI, open intervals use -Inf or +Inf as one limit.
806
807 If there is one single value the numeric interval is assumed
808 to be infinitely small (a scalar value).
809
810 \begin{itemize}
811 \item An open interval from the MJD 55100.0 and all later times:
812
813 \begin{lstlisting}
814 TIME= 55100.0 +Inf
815 \end{lstlisting}
816
817 \item A range of MJD values:
818
819 \begin{lstlisting}
820 TIME=55123.456 55123.466
821 \end{lstlisting}
822
823 \item An instant in time using Modified Julian Date:
824
825 \begin{lstlisting}
826 TIME=55678.123456
827 \end{lstlisting}
828 \end{itemize}
829
830 Time values are always UTC.
831 The TIME parameter is single-valued for \{sync\} requests and
832 multi-valued for \{async\} jobs.
833
834 The ucd of the TIME parameter is ``time.interval;obs.exposure'' and the unit is ``d''. The
835 datatype is ``double'' with an arraysize of 2,
836 and the xtype is, again, ``interval'' as defined in
837 DALI
838
839
840 \subsubsection{POL}
841 \label{sect:POL}
842
843 The POL parameter defines the polarization state(s) (Stokes)
844 to be extracted from the data.
845
846 \begin{itemize}
847 \item Extract the unpolarized intensity:
848 \begin{lstlisting}
849 POL=I
850 \end{lstlisting}
851 \item Extract the standard circular polarization:
852 \begin{lstlisting}
853 POL=V
854 \end{lstlisting}
855
856 \item The POL parameter is multi-valued; multiple values can be
857 included in a single request and all will be extracted.
858 Extract only the IQU components:
859 \begin{lstlisting}
860 POL=I
861 POL=Q
862 POL=U
863 \end{lstlisting}
864 \end{itemize}
865
866 The POL is multi-valued for both \{sync\} and \{async\} requests.
867 Unlike general filtering parameters, all values of POL are
868 combined into a single filter; for example, if the request
869 includes the three values above, the job would generate one
870 result with some or all of these polarization states (per
871 combination of ID and other filtering parameters).
872
873 The ucd of the POL PARAMETER is ``meta.code;phys.polarization'' and the unit is blank. The
874 datatype is ``char'' with arraysize ``*''.
875
876
877
878 \section{\{sync\} Responses}
879
880 All responses from the \{sync\} resource follow the rules for
881 DALI-sync resources, except that the \{sync\} response allows
882 for error messages for individual input identifier values.
883
884 \subsection{Successful Requests}
885
886 Successfully executed requests should result in a response
887 with HTTP status code 200 (OK) and a response in the format
888 requested by the client or in the default format for the
889 service.
890
891 If the values specified for cutout parameters do not include
892 any pixels from the target dataset/file, the service must
893 respond with HTTP status code 204 (No Content) and no
894 response body.
895
896 The service should set the following HTTP headers to the
897 correct values where possible.
898
899 \begin{tabular}{rr}
900 \sptablerule
901 Content-Type&mime-type of the response\cr
902 Content-Encoding&encoding/compression of the response (if applicable)\cr
903 \sptablerule
904 \end{tabular}
905
906 Since the response is usually dynamically generated, the
907 Content-Length and Last-Modified headers cannot usually be
908 set.
909
910
911 \subsection{Errors}
912
913 The error handling specified for DALI-sync resources applies
914 to service failure. Error documents should be text using the
915 text/plain content-type and the text must begin with one of
916 the following strings:
917
918 \begin{table}[h]
919 \begin{tabular}{rr}
920 Error&General error (not covered below)\cr
921 AuthenticationError&Not authenticated\cr
922 AuthorizationError&Not authorized to access the resource\cr
923 ServiceUnavailable&Transient error (could succeed with retry)\cr
924 UsageError&Permanent error (retry pointless)\cr
925 \end{tabular}
926 \caption{error messages with their meaning}
927 \end{table}
928
929 \section{\{async\} Responses}
930
931 The \{async\} resource conforms to the DALI-async resource
932 description, which means the job is a UWS job with all the
933 job control features available. All result files are to be
934 listed as children of the UWS results resource. The service
935 provider is free to name each result.
936
937 \appendix
938
939 \section{Changes from Previous Versions}
940
941 \subsection{Changes from WD-SODA-1.0-20151212}
942
943 \begin{itemize}
944 \item POS is now unitless
945 \item Aligned parameter UCDs with what is in Obscore
946 \item Removed gratuitous xtypes.
947 \end{itemize}
948
949 \subsection{Changes from WD-SODA-1.0-20151120}
950
951 Change the name of the protocol. Suppression of SELECT and COORD. xtype description are in DALI. Reference to this has been added.
952
953 \subsection{Changes from WD-AccessData-1.0-20151021}
954
955 Added general introduction on PARAMETER description to
956 section 3. Modified SELECT and COORD sections in order to
957 detach them from SimDal. Added Appendix on xtype description
958 with BNF syntax.
959
960 \subsection{Changes from WD-AccessData-1.0-20140730}
961
962 \begin{itemize}
963 \item Removed REQUEST parameter since the DAL-WG decision to not
964 include it when there is only one value.
965
966 \item Clarified that ID and filierting parameters are single
967 valued for \{sync\} and multi-valued for \{async\}, wth POL
968 being multi-valued but still being treated as a single
969 filter.
970 \end{itemize}
971
972 \subsection{Changes from WD-AccessData-1.0-20140312}
973
974 This is the initial document version.
975
976 \bibliography{ivoatex/ivoabib}
977
978 \end{document}

msdemlei@ari.uni-heidelberg.de
ViewVC Help
Powered by ViewVC 1.1.26