\documentclass[11pt,a4paper]{ivoa} \input tthdefs \usepackage{todonotes} \newcommand{\dousecase}[2]{% #1~\ref{uc:#2}} \newcommand{\Usecase}[1]{% \dousecase{Use case}{#1}} \newcommand{\usecase}[1]{% \dousecase{use case}{#1}} \title{The IVOA Source Data Model} \ivoagroup{Data Model} \author[http://wiki.ivoa.net/twiki/bin/view/IVOA/JesusSalgado]{Jesus Salgado} \author[http://wiki.ivoa.net/twiki/bin/view/IVOA/GerardLemson]{Gerard Lemson} \author[http://wiki.ivoa.net/twiki/bin/view/IVOA/MarkusDemleitner]{Markus Demleitner} \editor{TBD} % \previousversion[????URL????]{????Funny Label????} \previousversion{This is the first public release} \begin{document} \begin{abstract} Catalogues of astronomical objects make up an important part of the Virtual Observatory (VO). Almost since the beginning of the VO, the Simple Cone Search (SCS) standard provided a simple model for such data. The Source Data Model refines and formalises the data model implied by SCS with the overall goal of enabling a greater degree of automation in processing such catalogues. This standard derives requirements on the source data model from a set of use cases and then discusses the data model derived in turn from the requirements. \end{abstract} \section*{Conformance-related definitions} The words ``MUST'', ``SHALL'', ``SHOULD'', ``MAY'', ``RECOMMENDED'', and ``OPTIONAL'' (in upper or lower case) used in this document are to be interpreted as described in IETF standard RFC2119 \citep{std:RFC2119}. The \emph{Virtual Observatory (VO)} is a general term for a collection of federated resources that can be used to conduct astronomical research, education, and outreach. The \href{http://www.ivoa.net}{International Virtual Observatory Alliance (IVOA)} is a global collaboration of separately funded projects to develop standards and infrastructure that enable VO applications. \section{Introduction} The Simple Cone Search standard SCS \citep{std:SCS} very early in the development of the Virtual Observatory introduced a simple but effective model for describing a catalogue of astronomical sources, consisting of \begin{enumerate} \item A string-valued identifier. \item A ``main'' position, given as ICRS Right Ascension and declination in two floating-point fields. \item Arbitrary further information. \end{enumerate} The individual items are identified through Unified Content Descriptors \citep{std:UCD}. To enable limited machine interpretation of the further information, SCS relies on the mechanisms defined by VOTable \citep{std:VOTable}, in particular again UCDs, but also units, description, etc. While SCS has served the VO community well over more than a decade, several important use cases are not sufficiently covered by it, and the limited amount of annotation precludes many applications that require machines to have a deeper understanding of the catalogue's contents. This standard intends to supply the missing elements. We begin by sketching the motivating use cases and derive requirements from them in sect.~\ref{sect:ucreq}. Based on this analysis, in sect.~\ref{sect:dm} we give the VO-DML based definition of the SourceDM components. We conclude with considerations on phasing in SourceDM usage and possibilities to automatically infer rudimentary SourceDM annotation from legacy VOTable annotation.\todo{Complete the document structure} \subsection{Role within the VO Architecture} \begin{figure} \centering \includegraphics[width=0.9\textwidth]{archdiag.png} \caption{Architecture diagram for SourceDM} \label{fig:archdiag} \end{figure} Fig.~\ref{fig:archdiag} shows the role SourceDM plays within the IVOA architecture \citep{note:VOARCH}. It depends on the following standards: \begin{bigdescription} \item[VO-DML \citep{prestd:VODML}] SourceDM is written in VO-DML, a standard language to define data models, essentially providing an easy-to-process subset of the common modelling language UML sufficient to express data models useful in the Virtual Observatory context. \item[Mapping \citep{prestd:Mapping}] The Mapping document defines how to actually annotate catalogues written in VOTable (or possibly other formats in the future) with the concepts defined in this document. \item[PhotDM \citep{std:PhotDM}] The Photometry data model is re-used by SourceDM to express photometric quantities. \end{bigdescription} \section{Use Cases and Requirements} \label{sec:ucreq} \subsection{Use Cases} \label{sect:usecases} % please label use cases as uc:hyphenated-form-of-header Since a full model for an astronomical source would presumably result in a model of a large part of astronomy and physics, it is important to be guided by use cases when delimiting the scope of the model and formulating requirements on the model. This subsection will list the use cases the community agreed upon.\todo{I suspect we want some non-use cases, too, to avoid repetitive discussions about unrealistic wishes. MD} We use a ``want to'' form for the use cases, with two parties involved: Data providers who annotate their catalogues, and authors of programs or libraries (``clients'') dealing with the annotated data. \subsubsection{Standard Locations} \label{uc:standard-locations} Client authors want to to take SourceDM-annotated tables and easily provide plots of the locations of the contained objects in charts or over astrometrically calibrated images, preferably without having to implement a large number of coordinate transforms; this is essentially what tools can do right now with the ICRS positions given in SCS responses. \subsubsection{Identities} \label{uc:identities} Catalogue annotators want to express whether or not two instances (``rows'') refer to the same physical object on the sky. Client authors want to be able to match known-identical sources, for instance to aggregate information from several different catalogueues. \subsubsection{Native Frames} \label{uc:native-frames} Catalogue annotators want to give coordinates in their catalogue's native frames and for whatever epoch is convenient for them in order to achieve maximum precision, avoid hard-to-control error propagation in transformation requiring additinal information, and keep provenance simple. This becomes more important as we have more and more data noticeably distant from the J2000 epoch most of today's major surveys are close to, and more and more data precise enough that for a large number of objects proper their motions since J2000 become relevant for daily work. \subsubsection{Precise Astrometry} \label{uc:precise-astrometry} Client authors want to recognise all information in catalogues relevant to precision astrometry (e.g., distances or radial velocities as necessary in foreshortening calculations). This also involves the position of the observer, at least for a few standard reference positions. Of course it would be beneficial to be able to express actual observer positions, but in order to keep the model manageable, we suggest deferring this to provenance. \subsubsection{Crossmatching} \label{uc:crossmatching} Client authors want to implement smart crossmatching schemes between different catalogues. This includes reducing catalogues to common reference frames and epochs, but also unifying non-positional source characteristics (e.g., photometry). In the presence of crossmatching schemes based on principled statistical approaches, they want to identify data aiding them in the estimation of the relevant distributions. \subsubsection{Combining Data} \label{uc:combining-data} Following a crossmatch (or just within a single table), tool authors want to aggregate various pieces of information into new data points or products. Examples include: \begin{itemize} \item Colours or even SEDs from photometry points \item Proper motions or even orbits from positional measurements \item Time series from, say, radial velocity measurements \end{itemize} \subsubsection{Automatic Errors} \label{uc:automatic-errors} Where catalogues contain the necessary information, tool authors want to provide sound error estimates for derived quanities as discussed in \ref{uc:combining-data}. \subsubsection{Annotation of Existing Catalogues} \label{uc:annotation-of-existing-catalogues} Catalogue annotators do not want -- and indeed, in general cannot -- change catalogue data. SourceDM must therefore be flexible enough to enable the meaningful annotation of a large majority of the catalogues already in the VO. \subsubsection{Standard Properties are Annotated} \label{uc:standard-properties-are-annotated} Client authors want to find scientifically relevant source properties easily. These include \begin{itemize} \item source type \item spectral class \item mass \item physical size \item angular size \item redshift \end{itemize} \subsection{Requirements} % please reference use cases with \Usecase{hyphenated-form} (or % \usecase within a sentence) This section derives concrete requirements from the use cases collected in sect.~\ref{sect:usecases}. Some requirements are formulated using ``should'' phrases. ``Should'' as used in such requirements is not to be understood in the sense of RFC2119. \subsubsection{General Requirements} Most of the use cases of course require basic metadata necessary to interpret physical quanitites, in particular their units. \Usecase{identities} requires a concept of a physical object that a source refers to. It also means that in relational representations of collections of SourceDM instances, the physical object's identifier should not be the primary key.\todo{This may be wrong; perhaps we should mandate one instance per object per collection. I'm happy to strike this if it turns out to be dumb. MD} Regarding \usecase{standard-properties-are-annotated}, UCDs may be sufficient to annotate the concepts listed. Thus, at this point we derive no requirements from this use case. \subsubsection{Requirements on Space and Time Metadata} \Usecase{standard-locations} suggests that SourceDM instances should maintain the SCS practice of including ICRS positions wherever possible. Given \usecase{precise-astrometry}, it is highly desirable that these positions are nevertheless properly annotated, in particular with information on the epoch for which the position is pertinent, and where available the reference position. \Usecase{crossmatching} means that the following spatial concepts should be present in the model (even though of course many catalogues cannot fill all of them): \begin{itemize} \item Positions (given the context, spherical positions are probably sufficient in the model) \item Derivatives of positions (i.e., proper motions) \item Errors in the positions and their derivatives \item Epochs of the positions \end{itemize} In view of \usecase{precise-astrometry} together with \usecase{native-frames}, SourceDM should enable specifying frames with enough detail that sufficiently capable clients can use native frames for crossmatching. Less capable clients could still crossmatch at least partially using the standard locations from \usecase{standard-locations}. What this means in particular is that a single source should admit multiple positional specifications. In order to not leave the set of SourceDM-representable instances in \usecase{combining-data}, SourceDM should allow that these multiple specifications refer to different epochs. \Usecase{precise-astrometry} requires the third dimension, i.e., \begin{itemize} \item distance \item its derivative \item errors on both quanitites \end{itemize} Taking \usecase{annotation-of-existing-catalogues} into account, various forms of distance specification need to be admitted; in particular, many important catalogues give parallaxes rather than distances. Also note that the derivative of the distance, the radial velocity, will typically given in spatial coordinates even if the distance is given as (angular) parallax. In extragalactic astronomy, redshift is frequently used as a proxy for distance, and it is of course also a measure for the radial velocity. We still propose to not allow cosmological redshifts for distance or the derivative of the distance. In the nearby universe, the presence of a random motions (which might indeed be the actual objectiv a a catalog) makes this undesirable, in the more distant universe the interpretation of the term distance becomes difficult and should be avoided in favour of the redshift itself. Of course, where authors actually give redshift-based distances, these would be annotated as such. One consequence of \usecase{identities} is that SourceDM should allow statements of the type ``this row corresponds to source \emph{identifier-cat1} in catalogue 1 and to source \emph{idenifier-cat2} in catalogue 2''. \Usecase{automatic-errors} requires suffient information on the errors. In the astrometric case, this in particular concerns the mean epochs, i.e., the epoch at which the covariance matrix of positions and proper motions becomes diagonal. The mean epochs in general are different on two coordinates. \subsection{Requirements on Photometry} Lorem ipsum.\todo{Can we steal something from the PhotDM here?} \appendix \section{Changes from Previous Versions} No previous versions yet. % these would be subsections "Changes from v. WD-..." % Use itemize environments. \bibliography{ivoatex/ivoabib} \end{document}