ViewVC logotype

Contents of /trunk/projects/grid/notes/schema-versioning/schemaVersioning.tex

Parent Directory Parent Directory | Revision Log Revision Log

Revision 3142 - (show annotations)
Wed Oct 28 14:45:14 2015 UTC (4 years, 9 months ago) by harripa
File MIME type: application/x-tex
File size: 15641 byte(s)
clean up with Markus's feedback
completed the location on IVOA site section.
1 \documentclass[10pt,a4paper]{ivoa}
2 \input tthdefs
4 \usepackage{color}
5 \usepackage{listings}
6 \lstloadlanguages{sh,make,[latex]tex,XML,XSLT}
7 \lstset{flexiblecolumns=true,numberstyle=\small,numbers=left,
8 identifierstyle=\texttt}
11 \usepackage{color}
12 \definecolor{gray}{rgb}{0.4,0.4,0.4}
13 \definecolor{darkblue}{rgb}{0.0,0.0,0.6}
14 \definecolor{cyan}{rgb}{0.0,0.6,0.6}
16 \lstset{
17 basicstyle=\ttfamily,
18 columns=fullflexible,
19 showstringspaces=false,
20 commentstyle=\color{gray}\upshape
21 }
23 \lstdefinelanguage{XML}
24 {
25 morestring=[b]",
26 morestring=[s]{>}{<},
27 morecomment=[s]{<?}{?>},
28 stringstyle=\color{black},
29 identifierstyle=\color{darkblue},
30 keywordstyle=\color{cyan},
31 % morekeywords={xmlns,version,type}% list your attributes here
32 }
35 \usepackage{todonotes}
37 \usepackage[utf8]{inputenc}
39 \definecolor{texcolor}{rgb}{0.4,0.1,0.1}
40 \newcommand{\texword}[1]{\texttt{\color{texcolor} #1}}
42 \iftth
43 \newcommand{\BibTeX}{BibTeX}
44 \fi
46 \iftth
47 \newcommand{\comicstuff}[1]{
48 \begin{html}<span class="comic">#1</span>\end{html}}
49 \else
50 \newcommand{\comicstuff}[1]{(HTML exclusive material)}
51 \fi
52 \customcss{custom.css}
54 \title{XML Schema Versioning Policies}
56 \ivoagroup{Standards and Processes}
58 \author[http://www.ivoa.net/cgi-bin/twiki/bin/view/IVOA/PaulHarrison]{Paul Harrison}
59 \author[http://www.ivoa.net/cgi-bin/twiki/bin/view/IVOA/MarkusDemleitner]{Markus Demleitner}
60 \author[http://www.ivoa.net/cgi-bin/twiki/bin/view/IVOA/BrianMajor]{Brian Major}
61 \author[http://www.ivoa.net/cgi-bin/twiki/bin/view/IVOA/PatDowler]{Pat Dowler}
63 \editor{Paul Harrison}
65 \previousversion{This is the first public release}
67 \begin{document}
69 \SVN$Rev$
70 \SVN$Date$
71 \SVN$URL$
73 \begin{abstract}
74 This note describes the recommended practice for the evolution of IVOA
75 standard XML schemata that are associated with IVOA standards. The criteria for
76 deciding what might be considered major and minor changes and the policies for dealing with each case are described.
77 \end{abstract}
81 \section*{Acknowledgments}
83 The content of this note is derived from discussions that occurred in a splinter
84 session at the June 2015 IVOA interoperability meeting in Sesto, Italy.
86 \section{Introduction}
88 Many of the standard protocols and data models developed by
89 the International Virtual Observatory Association (IVOA) have used XML
90 \citep{std:XML} for message or object serialization. The structure of
91 these XML files has
92 usually been constrained using the XML schema definition language
93 \citep{std:XSD}, or XSD for short.
94 The particular schema that has been associated with a standard
95 is defined by the ``target namespace'' (hereafter refered to as simply ``the
96 namespace'') of the schema - the namespace identifier itself is a URI that
97 typically has a form that contains the version number of the standard. There
98 exist in many programming languages XML parsers that can use the XSD schema
99 automatically to do a strong check whether an instance of an XML
100 document conforms to the given schema. If the XML document does conform to the
101 structure defined by the schema then it is known as ``valid'' and even a small
102 deviation from the specified structure will mean that the XML instance is
103 ``invalid''. This strong check of validity is extremely useful in the context of
104 interoperating services and clients within the VO as it guarantees that both
105 sides of an interaction will agree on the structure of the document and
106 hence its interpretation. To maintain
107 this behaviour, the conventional XSD practice is to give the schema a new
108 identity (namespace) if any changes are made to it so that both client and
109 server can agree on the exact version of a schema that they are using for
110 checking validity of an XML instance.
112 Once a VO service has been standardised there will typically be a growing number
113 of clients that are coded against the particular version of the
114 schema - any changes to the definition of the schema that is subsequently used
115 by a new version of the service to create instance documents will result in
116 immediate classification as ``invalid'' by the clients that have not themselves being
117 updated to use the new schema.
118 As explained above, in general this property is desirable for guaranteeing interoperability, but it does limit
119 the ability even to correct errors in the original schema definition without
120 causing disruption to the deployed clients.
122 A related problem occurs because to namespace-aware XML parsers, the
123 element name is a tuple of the namespace URI and the tag content. For
124 instance, in the XML document
126 \begin{lstlisting}[language=XML]
127 <doc xmlns="http://example.com/1.0"/>
128 \end{lstlisting}
130 the full name of the element is conventionally written as
131 \verb|{http://example.com/1.0}doc|, and clients will expect this full
132 form, whether or not they perform schema validation. As soon as the
133 namespace changes, any client expecting this element name will no longer
134 understand anything in the document. Rather typically, however, the
135 client would still be able to make sense of the document as far as it is
136 relevant to the client if it ignored the namespace part of the element
137 name. In consequence, many clients started discarding the namespace
138 part entirely, which then leads to new interoperability problems, for
139 instance, when elements from a different namespace are embedded within
140 such a document. Hence it is desirable to limit namespace changes to
141 cases when legacy clients would definitely break.
143 This note describes the
144 circumstances in which it is permissible to make changes to a schema and not
145 change its namespace.
148 \section{Schema Versioning}
149 It is the case that two XML instances are formally regarded by XML
150 conventions as not equivalent if the only textual difference between them is
151 that the namespace declaration is not the same. We
152 can use the observation that simply removing the namespace definition from both instances
153 would make the instances both textually the same and equivalent in the strict
154 XML sense to inform the class of changes can be to the schema definition without
155 necessarily needing to change the namespace, and how a client should treat such
156 changes.
157 \subsection{Minor changes}
158 In general the class of changes that might be considered minor are those
159 which allow legacy clients (i.e. without rewriting) to keep functioning with
160 content produced against the new schema.
161 Specific structual changes to the schema that allow that goal to be achieved
162 include:
164 \begin{itemize}
165 \item Not removing concepts (i.e. elements or attributes) from the old
166 schema.
167 \item Making any new concepts that are introduced optional.
168 \end{itemize}
169 Even with the restrictive conditions above it is still necessary that any
170 consumer of XML instance documents takes the approach that it does not do strict
171 schema validation against the version of the schema that it knows about, but
172 rather ignore everything that it does not understand. This approach is allowable
173 because any new concepts are optional even for consumers of the XML instance
174 that are aware of the latest version of the schema, and so clients cannot use
175 this information for fundamental changes in the behaviour of an IVOA protocol.
176 In other words the IVOA protocol remains backwards compatible and would only
177 warrant a ``point change'' in the standard version.
179 Conversely if the conditions above cannot be met by an evolution of an IVOA
180 standard then it is a indication that the standard is undergoing a ``major
181 change'' and backwards compatibility may be broken. Such a change would
182 involve both a change in the first part of the standard document's version
183 number as well as a change in the version used in the schema
184 namespace.
185 \subsubsection{Determining if the changes are indeed minor}
186 Although the conditions outlined above for minor changes should generally be
187 strictly adhered to in the design of minor extensions to standards, there are
188 occasions where a new version of a standard might try to correct an error made
189 in a previous version. For example a certain construct could have been
190 schema valid which should not have been allowed in a correctly authored
191 schema that expressed the intentions of the standard. In this case it is likely
192 the the correction would break the first of the above conditions, but hopefully
193 clients would not be written to expect the unintentionally valid construct and
194 would not be unduely impacted by the change. It would be up to the IVOA
195 Technical Coordination Group (TGC) to determine the likely impact of such a
196 change and whether the violation of the constraints for a minor change would be
197 allowable in such circumstances.
199 \subsection{Indicating the version number}
201 When allowing minor changes to the schema without changing the
202 namespace it becomes imperative that the the minor version number of the schema
203 is communicated via a means other than the namespace. This is true for
204 both instance documents and schema documents.
206 \subsubsection{Version numbers in instance documents}
208 It is important that
209 clients are able to know which version of the schema was used to create an XML
210 instance. A common use case envisioned is that in service responses,
211 this information can be used to discover a service's ability to
212 correctly respond to newly specified parts of the protocols.
214 Hence, in service responses root elements must have
215 a \xmlel{version}
216 attribute exactly giving the value of the correspoing XSD file's
217 \xmlel{version} attribute.
219 When global elements from a different schema are included within other
220 XML files, each of these global elements should have a version attribute
221 of its own. Where the discovery of extended capabilities is not a
222 likely use case, such version attributes may be left out.
224 \subsubsection{Version numbers in schema files}
225 It is of similar importance to software writers to know exactly the version of
226 the schema that they are using. In the XSD definition there is an appropriate
227 \xmlel{version} attribute on the top level \xmlel{<schema>} element that can be
228 used for this purpose.
230 \begin{lstlisting}[language=XML]
231 <xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema"
232 targetNamespace="http://www.ivoa.net/xml/UWS/v1.0"
233 xmlns:uws="http://www.ivoa.net/xml/UWS/v1.0"
234 xmlns:xlink="http://www.w3.org/1999/xlink"
235 elementFormDefault="qualified"
236 attributeFormDefault="unqualified"
237 version="1.1-PR-20150626"
238 >
239 \end{lstlisting}
241 This version attribute is not used formally by the schema validation machinery
242 but can be used as desired here to indicate a precise version of the schema
243 given that the namespace is only used to differentiate between major revisions
244 of the schema.
248 \section{Hosting the Schema on the IVOA Web Site}
249 As mentioned in the introduction the schemata are typically associated with a
250 particular version of a standard and so are typically included in an appendix of
251 the standard. In addition they are located on the ivoa website
252 \url{http://www.ivoa.net/xml} to allow software authors interactively to
253 obtain the latest version directly as a file.
255 The availability of the schemata via the IVOA web site also
256 allows XML parsers to be instructed to fetch the schema automatically when
257 validating an XML instance. This is done using the \xmlel{xsi:schemaLocation}
258 attribute which pairs a namespace with the URL of a schema that has definitions
259 in that namespace. Although there is no particular necessity to do so, it has
260 been standard practice to use namespaces that have a one-to-one correspondence
261 with a location on the IVOA web site. The advantage of this approach is that the
262 namespace is ``owned'' by the IVOA and there is therefore much less chance of
263 accidental clashes of namespaces.
265 Thus a current schema is usually available at two URLs on the IVOA site;
266 \begin{enumerate}
267 \item at a URL that corresponds to the namespace \newline e.g.
268 \texttt{http://www.ivoa.net/xml/UWS/v1.0}
269 \item at a URL that corresponds to the filename of the schema \newline e.g.
270 \texttt{http://www.ivoa.net/xml/UWS/UWS-v1.0.xsd}
271 \end{enumerate}
273 As this note now recommends that there are potentially many minor versions of a
274 schema all in the same namespace the actual file than the first from of the URL
275 above points to must be updated when a new minor version is published. The
276 various minor versions of a particular schema should be available at the
277 ``filename'' style URLs of the second kind. It should be noted that the
278 actual file pointed to by the ``namespace'' style URL should only be updated to
279 point to the new minor version when the associated standard has reached the
280 ``recommendation'' stage.
282 \subsection{Client use of schema}
283 Using this strategy means that client software that simply uses validation with
284 the parser utilizing this schema location hint will always pick up the latest
285 version of the schema, which should mean that the instance document will always
286 be valid assuming that the server has created a strictly valid document. Note
287 that this still applies even if the server is using an older (minor) version of
288 the schema, as new versions are only allowed to add optional features to the
289 schema.
291 It is often the case that client software is written to use a local copy of the
292 schema to avoid the overhead of continually downloading the schema. In this case
293 it is possible that the XML instance document will not be valid against the
294 local schema if the server is working against a newer minor version of the
295 schema - as it could have added an new optional element into the XML instance.
296 It is for this reason that the client side of an IVOA service should be
297 forgiving in ignoring elements that it is not expecting. It is possible for the
298 client to recognise that the XML instance is valid against a newer version of
299 the schema by comparing the version attribute of the root element of the XML
300 instance against the version attribute in the local XSD schema document.
302 \section{Summary of Recommendations}
303 This section summarizes the main recommendations contained within this document.
305 \begin{itemize}
306 \item The namespace URI should contain only the associated standard major number,
307 and therefore does not change when a minor version increment occurs.
308 e.g.\ \texttt{http://www.ivoa.net/xml/ABC/v1}. However, existing namespace URIs
309 which contain a ``\texttt{.0}'' minor increment suffix should not be changed to
310 conform to this rule.
312 \item Include a \xmlel{version} attribute in the top level element to allow
313 client version discovery - this version number should include the full standard version
314 number including the minor version increment. -- 1.1
316 \item Set the \xmlel{version} attribute of the \xmlel{<schema>} element in the
317 XSD to be equal to the full standard version including the minor version
318 increment and document status and version - e.g.\ 1.1-WD-20150607
320 \item Minor version changes should not break clients - i.e. in general they
321 should only add new elements/attributes, not remove formerly valid content -
322 correspondingly, this means that clients should quietly ignore things that they
323 do not know about in the XML - i.e. they should not automatically issue an error
324 if schema validation fails (though of course if they can determine that the
325 validation error is because of a missing required element then they should
326 issue an error).
327 \item In contrast servers must produce strictly valid documents, and service
328 validators must test strict validity against the relevant schema (discovered
329 from the namespace and the version element).
330 \end{itemize}
335 \appendix
338 \section{Changes from Previous Versions}
340 No previous versions yet.
341 % these would be subsections "Changes from v. WD-..."
342 % Use itemize environments.
345 \bibliography{ivoatex/ivoabib}
348 \end{document}


Name Value
svn:keywords Date Rev URL

ViewVC Help
Powered by ViewVC 1.1.26