Change Log Proposal

The associated version (http://open-services.net/wiki/core/TrackedResourceSet-2.0/) of the Change Log proposal is here: OslcTrackedResourceSet

In addtion to being a significantly improved document that now clearly describes the protocol, there are also some design changes in this version:

1. Added a root resource, "TrackedResourceSet", to package a Change Log along with its corresponding set of initial (now called Base) resources.
2. A "cutoff event" is provided on the first page of the Base resources. Provides the sync point between the Base and its corresponding Change Log.
3. Renamed the change log entry "at" predicate to "order".
4. Added an "identifier" property to change log entries. Needed because server roll-back may reuse "order" values previously allocated.
5. Renamed the Change Log "priorChangeLog" predicate to "previous".
6. Changed the namespace for the resources in this specification from http://open-services.net/ns/core/log# to http://open-services.net/ns/core/trs#.
7. Now calling this the Tracked Resource Set Specification (instead of Change Log Specification). Is this a good name? Other suggestions?

Earlier Discussions

The previous Change Log proposal is described in the attached document:

http://open-services.net/pub/Main/IndexingProposals/OSLC_indexing_0316.doc

It includes changes to reflect discussion and decisions made during the first round of review, including the following:

1. A section to describe the motivating use case.
2. Description of a formal "Indexing Profile" which defines the capabilities that a service provider MUST support in order to be indexable.
3. Proposed formal scope of indexing/changeLog.
4. Change Log entry timestamps changed to sequence numbers.
5. Change Log entries can optionally be referenced URI-addressable resources.

Current Proposal Status

We are very close to finalizing the OSLC ChangeLog? proposal. The last three issues, which are still being discussed are:

1. Indexing/ChangeLog resources/scope

Latest email: http://open-services.net/pipermail/oslc-core_open-services.net/2011-March/000898.html

Summary:

1. Indexable resources are simply "public resources", i.e., the resources that the service provider does not consider to be internal implementation artifacts.
2. Public resources are exposed by setting oslc:usage property on one or more Query Capability to a special value (e.g., "http://open-services/ns/core/log#resources").
3. We need to document clearly that we require only the queryBase. No actual query syntax needs to be supported.

2. ChangeLog? Pagination (also URI-addressable ChangeLog? entries)

Latest email: http://open-services.net/pipermail/oslc-core_open-services.net/2011-March/000893.html

Summary:

1. OSLC paging is not well suited for paging an rdf:list, and therefore not the right mechansim for ChangeLog? paging. ChangeLog? will instead use it's own property (oslc-log:continuation, or maybe oslc-log:next?) for paging.
2. Reject the requirement for individually-addressable entries. They are not useful unless we also design a way to automatically retrieve the referenced resources.

Latest example from email thread:

<https://.../ChangeLog>
oslc-log:changes (
...
]) ;
oslc-log:continuation <https://.../ChangeLog_1> .

Should we include a type and make some other syntactic changes? For example:

<https://.../ChangeLog>
a oslc-log:ChangeLogSegment ;
oslc-log:changes (
...
]) ;
oslc-log:nextSegment <https://.../ChangeLog/1> .

3. Better name for OSLC indexing profile

Latest email: http://open-services.net/pipermail/oslc-core_open-services.net/2011-March/000896.html

Summary:

The following suggestions are on the table:

1. Observer Profile
2. Notification Profile
3. Change Log Profile
4. Content Tracking Profile
5. Service Tracking Profile
6. Active RDF Index Source (ARIS) Profile

Also, what should the URI be for the components of this proposal:

1. oslc-log = "http://open-services/ns/core/log"?
2. oslc-aris = "http://open-services/ns/core/aris"?
3. other?

Previous discussion items from OSLC_indexing_0316.doc

1. Section 1.1: Is Collaborative Lifecycle Management (CLM) an OSLC term? If this spec is intended for publishing at OSLC, we need to find an OSLC way of saying this.
2. Section 1.2.1: What is an "indexable resource"? How are they found? I (Martin) don’t think we should define “indexable resources”. If you make it public, we will index it, and other clients may do other things with it. People do not define “indexable HTML pages”. The page is either public and present or it isn’t. You don’t want providers speculating about what client might come along to exploit their data. You want the usages to be limited only be the imagination of people who might write clients (like indexers), not by the imagination of the providers. The only decision a ServiceProvider? should make is “is this public or not”. If it is intended to public, make sure it’s in some ServiceProvider? .
3. Section 1.2.2: What is a more general name than "Indexing Profile"? It’s not just for indexers, it’s for any client that wants to see content and track changes to content.
4. Section 1.2.2: Should we use a different namespace than "oslc:" for ChangeLog? ? Proposal: oslc-log: -> http://open-services.net/ns/log # or oslc-change: -> http://open-services.net/ns/change# ?
5. Rename oslc:create, oslc:update, oslc:delete to oslc:CreateEvent, oslc:UpdateEvent, oslc:DeleteEvent (i.e., uppercase nouns). Another option is oslc-log:Creation, oslc-log:Modification, oslc-log:Deletion
6. Section 1.2.2: ChangeLog? with references to change entries: Is this one resource or 4? If it’s only one resource, are we saying that the triples for the change events must be in the changelog (or changelog page) representation even though the changeEvents have their own URLs? Are the same triples available by doing a GET on the individual event URLs? Need clarification.
7. Should we make rdf:list entries URI addressable, instead of the change entries themselves? Is there a better way to paginate an rdf:list? See email thread with subject: "Pagination".
8. Also regarding pagination: if a provider uses 'unstable paging' of large query results, as most probably will, what guarantees are there that a query will give you all resources at least once? The description of unstable paging in the core spec just says that the resources might change as you read each page of the list, but do not mention the integrity or consistency or lack thereof for the result set itself. If the 'next page' link might return a page with a resource skipped over (perhaps because some resources from a previous page have been deleted, and so a resource would now appear on an earlier page), then the initial population of an index might be incomplete. Should the core spec provide a general guarantee that paged results cannot skip resources, or should this belong in the indexer profile (or whatever we call it), or should an indexer use some other techniques to check for missing resources?

Example from "Pagination" Email (issue #7, above)

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> PAGE 1
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
@prefix oslc: <http://open-services.net/ns/core#> .
@prefix xsd: <http://www.w3.org/2001/XMLSchema#> .
@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
@prefix : <https://.../ChangeLog#>.

<https://.../ChangeLog?oslc.paging=true>
oslc:nextPage <https://.../ChangeLog?pageno=2> .

<https://.../ChangeLog> oslc:changes :b1.
:b1 rdf:first
[ a oslc:create ;
oslc:changed <https://.../com.ibm.team.workitem.WorkItem/23> ;
oslc:at "103"^^xsd:int
] ;
rdf:rest :b2 .
:b2 rdf:first
[ a oslc:update ;
oslc:changed <https://.../com.ibm.team.workitem.WorkItem/22> ;
oslc:at "102"^^xsd:int
] ;
rdf:rest :b3 .
:b3 rdf:first
[ a oslc:delete ;
oslc:changed <https://.../com.ibm.team.workitem.WorkItem/21> ;
oslc:at "101"^^xsd:int
] ;
rdf:rest :b4 .

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> PAGE 2
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
@prefix oslc: <http://open-services.net/ns/core#> .
@prefix xsd: <http://www.w3.org/2001/XMLSchema#> .
@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
@prefix : <https://.../ChangeLog#>.

<https://.../ChangeLog?pageno=2>
oslc:nextPage rdf:nil . <<<<<<<<< or omit the nextPage property entriely

:b4 rdf:first
[ a oslc:create ;
oslc:changed <https://.../com.ibm.team.workitem.WorkItem/20> ;
oslc:at "100"^^xsd:int
];
rdf:rest rdf:nil .

Older Versions of Change Log Proposal

The previous version of the proposal can be found in section 1.1 of OSLC_proposals.doc. The following items were discussed during the first round of review:

• Should change log clients be able to request changes since some date-time? For example, using a URI such as https://../ChangeLog?oslc.since=mmddyy.
  • The current thinking is that so long as the pages are inverse-time-ordered, clients have enough mechanism already to satisfy this need.
  • If page sizes turn out to frequently be much bigger than the needed changes, we think that adding an optimization like this in the future would be simple. Removing or changing it later would be harder.

• Might each change event in a change log be possibly a URI-addressable resource itself?
  • We might clarify that indexing clients need to be able and willing to accept this:
    
    @prefix oslc: <http://open-services.net/ns/core#> .
    @prefix xsd: <http://www.w3.org/2001/XMLSchema#> .
    
    <https://.../ChangeLog>
    oslc:changes (https://.../changeEvent/22 https://.../changeEvent/21 https://.../changeEvent/20).
    
    https://.../changeEvent/22 a oslc:create ;
    oslc:changed <https://.../com.ibm.team.workitem.WorkItem/23> ;
    oslc:at "2010-12-02T21:28:48.064Z"^^xsd:dateTime.
    
    https://.../changeEvent/21 a oslc:update ;
    oslc:changed <https://.../com.ibm.team.workitem.WorkItem/22> ;
    oslc:at "2010-12-02T21:25:57.299Z"^^xsd:dateTime.
    
    https://.../changeEvent/20 a oslc:delete ;
    oslc:changed <https://.../com.ibm.team.workitem.WorkItem/21> ;
    oslc:at "2010-12-02T21:22:54.72Z"^^xsd:dateTime.
  • Less clear is whether they need to be able and willing to accept this:
    
    @prefix oslc: <http://open-services.net/ns/core#> .
    @prefix xsd: <http://www.w3.org/2001/XMLSchema#> .
    
    <https://.../ChangeLog>
    oslc:changes (https://.../changeEvent/22 https://.../changeEvent/21 https://.../changeEvent/20).
  • The implication of accepting this is that the client is willing to perform a GET on each change event individually.
  • Another interesting issue might be the use of https. I believe that using https means that proxy servers can't cache changelog pages. Is that what we want to encourage?

• Is oslc:at being used as a sequence number and need not be a timestamp? Or, is oslc:at required to be the time at which the change occurred?
  • It really is just a sequence number. All that matters is that we can compare values to decide which is more recent. If we use integer, instead of date, do we need to worry about roll-over?

• There is a Talis RDF vocabulary for change set (http://n2.talis.com/wiki/Changesets) which includes some parts of this proposal. It includes the idea of a change to a resource, and additionally (and relevant to Olivier's follow-up) information on the nature of the change in terms of triples added/removed. It does not expressly deal with deletion or creation. (The Talis resource model doesn't have the idea of a log.)
  • How much of it, if any, could/should we reuse? Most of it has to do with the fine-grain resource change details, which we are currently wanting to keep out of this model. Steve Speicher suggested that we should use standard predicates where appropriate (e.g., dc:date instead of oslc:at). I was wondering if cs:subjectOfChange would a better replacement for oslc:resource (in the current proposal). Would this ChangeSet? model to be "standard" enough to justify reusing parts in our OSLC models?

• Another question is whether a ChangeLog? service is a "required OSLC service"?
  • Current thinking is that it is "highly recommended". A tool needs to implement ChangeLog? if it wants to be able to participate in a central RDF index.
  • A related question is, would it be easier for a tool to implement ChangeLog? than the OSLC query service? If so, could a query service leverage a ChangeLog? -driven index to implement/provide query?

• Other issues?