[oslc-core] TRS truncation of change log

[Joe Ross]

Yes, you got it exactly and explained it better than I did. Thanks!

I think the temporarily offline scenario is the most likely. This was
actually brought in discussion of our proposed change-log functionality
with a product team planning to synchronize their product repository with
our resource registry.

Joe

================================================
Joe Ross/Austin/IBM, joeross at us.ibm.com
Tivoli Autonomic Computing & Component Technologies
512-286-8311, T/L 363-8311



From:	Vivek Garg/Cupertino/IBM
To:	Joe Ross/Austin/IBM at IBMUS,
Cc:	oslc-core at open-services.net, "Oslc-Core"
            <oslc-core-bounces at open-services.net>, Matthew
            Jarvis/Lexington/IBM at IBMUS
Date:	05/02/2013 04:29 PM
Subject:	Re: [oslc-core] TRS truncation of change log


Hi Joe,

Below I have tried to replay my (and Matt Jarvis's ) understanding of the
issue you are trying to solve and your proposed solution:

Problem: It may be inefficient to detect truncated change logs in some
(edge?) cases
When processing a change log, a client may paginate through change log
segments (pages) looking for the last processed change event or the
cutoffEvent. For clients that have not recently processed a change log, it
may mean fetching a large number of change log segments to locate the last
processed or cutoffEvent. If the change event was found, the effort of
paginating through the pages is worth it.  But if we were to fail to locate
the change event after paginating all the pages, it is mostly wasted
effort.

Some real life scenarios are:
a. A client has been busy processing the base resource for a very long time
(e.g. for a week). While the base was being processed, the cutoffevent
change event (latest change event reflected in the base) has become a
truncation candidate. And server decided to truncate the change log
including the cutoffEvent.
b. A client has been offline for a long time e.g. 7 days. One online, the
client may paginate through all the change log pages, only to find that
change log is missing the last processed change event.

Solution:
We can provide a fast-fail detection for truncated change logs by including
(perhaps optionally) the lastEvent in the change log segments.

Does this accurately reflect the scenario you are thinking about?

Regards
Vivek





From:	Joe Ross/Austin/IBM at IBMUS
To:	oslc-core at open-services.net,
Date:	05/02/2013 02:24 PM
Subject:	[oslc-core] TRS truncation of change log
Sent by:	"Oslc-Core" <oslc-core-bounces at open-services.net>



The Tracked Resource Set specification, states the following:
      "To ensure that a new Client can always get started, the Change Log
      MUST contain the base cutoff event of the corresponding Base, and all
      Change Events more recent than it. Thus the Server is only allowed to
      truncate Change Events older than the base cutoff event. "



However, since processing of the change log would happen some time after
the base was read, it is possible that truncation happens between the time
that a client reads the base and the time that it starts processing the
change log. Truncation could also happen while a client is paging through
the change log. In that case, it would be useful for a client to know about
the truncation event before processing the entire change log, so that it
can switch gears and obtain a new base snapshot instead. It seems that it
might be useful if each page of the change log also included a lastEvent
property. If at some point during change log processing, lastEvent becomes
more recent than the cutoffEvent value that client knows, the client could
then abandon change log processing and obtain a new base snapshot instead.

Of course, we can add this property in our implementation, but seems it
might be useful as an addition to the spec.


================================================
Joe Ross/Austin/IBM, joeross at us.ibm.com
Tivoli Autonomic Computing & Component Technologies
512-286-8311, T/L 363-8311_______________________________________________
Oslc-Core mailing list
Oslc-Core at open-services.net
http://open-services.net/mailman/listinfo/oslc-core_open-services.net


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://open-services.net/pipermail/oslc-core_open-services.net/attachments/20130502/d0eb49d3/attachment-0003.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: graycol.gif
Type: image/gif
Size: 105 bytes
Desc: not available
URL: <http://open-services.net/pipermail/oslc-core_open-services.net/attachments/20130502/d0eb49d3/attachment.gif>