Attachments for Change Requests

Early Working Draft This is used to explore how to support attachments for CM ChangeRequest resources for a couple of scenarios. In being under development, it is NOT recommended

Motivation

Common use case is to have potentially many and possibly large attachments associated with a Change Request.

Things to consider:

• Other than the attachment (file), what properties exist for this attachment? There are typically some automatically added: creation date, owner, size, perhaps content type, id. Others are typically added: name, description. What of these really need to be specified to support the scenarios?
• What operations are needed on attachments: view, create, delete, update?
  • create, view are primary ones.
• Support for bulk / multiple upload of attachments? Seems like a nice feature but really required?
  • Primary need is a single attachment file upload / create
• Support in existing CM providers: need to consider limitations of mapping to existing models? For example: Attachments are typically considered resources that one of its property is the actual "file". Attachments have properties that can be modified individually and some that are system defined/computed.
• Need to know CM tool size limits upfront to prevent sending file too big and then getting rejected.

Some existing behavior:

• ClearQuest supports POSTing the actual content of a file or files to the ChangeRequest's attachment property URI. It accepts multipart content for multiple files and optionally a description property for the resources. Files can only be attached directly to a ChangeRequest.
• RTC supports POSTing the actual content of a file to the repository. A 2nd request is needed to create the "attachment descriptor" resource that links the ChangeRequest to the file attachment. There is a background job that cleans up any files that have not been linked after some point in time. It also allows multiple bindings per file.
• Rational Change doesn't provide a way to add attachments via its REST interface. Support in product is standard attachment.
• Bugzilla has REST support for Attachments as well. There are some additional properties that it has: encoding (base64 only one supported), comment, flags, is_obsolete, is_patch, is_private, update_token which seem to be "extra".

Supporting material:

• Attaching Source Scan results to an existing ChangeRequest
• Attaching Log file, screenshots, source file to an existing ChangeRequest from a minimal CM client
• V2 Technical Priorities
• OSLC Core Discussion on Attachments
• OslcCoreAttachments

Approach

The additions to CM proposed here are minimal, and the model is optimized for the most common scenarios: creating and viewing attachments. Other operations, including deleting and updating, can also be supported.

Conceptual Model

The additions to the conceptual model for CM consist of one new ChangeRequest property, and one new resource type, AttachmentList.

Resource ChangeRequest

ChangeRequest will have a have a new property linking each change request to its associated list of attachments.

Resource AttachmentList

AttachmentList is a new resource type used to catalog the attachments associated with a particular change request. Such a resource also acts as a factory for new attachments to that change request. In this model, each attachment represents the content (or file), itself, simplifying the attachment creation scenario.

• Name: AttachmentList
• Type URI http://open-services.net/ns/cm#AttachmentList

There is no restriction on the content of each attachment resource. For example, it could be an image, a document, a log file, or a patch. Since the attachment cannot be expected to contain metadata, a standard set of metadata for each attachment is included within the AttachmentList itself. Thus, the object of each oslc_cm:attachment statement also appears as the subject of statements using some or all of the following properties.

If resource paging is used for an AttachmentList, the oslc_cm:attachment statement referencing a particular attachment MUST be grouped together on the same page with the metadata statements about that attachment.

Metadata that is not read-only can be updated via a PUT on the AttachmentList.

Open questions

• Do we need both dcterms:title and dcterms:name (does that exist?)?
• Should dcterms:format use PURL media-type resources? What about parameters (e.g. charset)?
• Should we consider file properties vocabulary instead for format and size?
• What about dcterms:description (and would that require another non-standard header below)? Or a link to a comment, as suggested in the Mylyn scenario?

in some cases Files are attached along with a supporting Comment

Attachment Creation

An attachment is added to a ChangeRequest via a simple POST request to its AttachmentList resource. The entity body becomes the content of the attachment resource. The attachment is automatically associated with the change request via an oslc_cm:attachment statement in the AttachmentList. Metadata statements are also automatically added to the AttachmentList. All of these metadata values are assigned by the CM provider or can be determined from standard headers of the POST (e.g. dcterms:format and oslc_cm:contentSize are determined from Content-Type and Content-Length, respectively), with one exception: dcterms:title.

For this reason, a non-standard header is introduced: Content-Name ???. A CM consumer can include a Content-Name header in the attachment-creating POST to specify a name for the resource as part of that single request. This can be important as some CM systems require a name at the time the attachment is created. Different systems may have different requirements for valid attachment names, so the value of the Content-Name header should be interpreted as a hint in this context. If the given name can not be used as specified, it SHOULD be transformed into a valid name. If that is not possible or the header is not specified, an arbitrary value SHOULD be assigned. Failure due to an invalid name is undesirable because of the potentially large size of an attachment resource.

A Content-Name header SHOULD also be included by a CM provider in the response to a GET on an attachment resource. If a consumer wishes to store the content as a file, this value provides a hint as to the file name to use (subject, of course, to any file system restrictions). In the absence of a Content-Name header, the consumer may use the last segment of the resource's URI as a hint, or just choose an arbitrary file name.

Open questions

• Address deletion specifically: DELETE the attachment vs. PUT the AttachmentList with the oslc_cm:attachment statement removed?
• Address update: PUT on an existing attachment, as suggested in the Mylyn scenario?
• Consider adding existing resources (under the control of the provider or elsewhere) to the AttachmentList?

Sample Usage

Retrieve an Attachment for a Change Request

This involves three GET requests: the first to find the URI of the attachment list, the second to list attachments and their metadata, and the third to retrieve an attachment's content. In this example, there are two attachments associated with the change request.

Request #1

GET http://example.com/bugs/2314?oslc.properties=oslc_cm:attachments
Accept: text/turtle

Response #1

HTTP/1.1 200 OK
Content-Type: text/turtle
Content-Length: 183
ETag: "_87e52ce291112"

@prefix oslc_cm: <http://open-services.net/ns/cm#> .

<http://example.com/bugs/2314>
   a oslc_cm:ChangeRequest ;
   oslc_cm:attachments <http://example.com/bugs/2314/attachments> .

Request #2

GET http://example.com/bugs/2314/attachments
Accept: text/turtle

Response #2

HTTP/1.1 200 OK
Content-Type: text/turtle
Content-Length: 944
ETag: "_87e52ce293334"

@prefix dcterms: <http://purl.org/dc/terms/> .
@prefix oslc_cm: <http://open-services.net/ns/cm#> .

<http://example.com/bugs/2314/attachments>
   a oslc_cm:AttachmentList ;
   oslc_cm:attachment
      <http://example.com/bugs/2314/attachments/1>,
      <http://example.com/bugs/2314/attachments/2> .

<http://example.com/bugs/2314/attachments/1>
   dcterms:title "screenshot.png" ;
   dcterms:identifier "1" ;
   dcterms:format <http://purl.org/NET/mediatypes/image/png> ;
   oslc_cm:contentSize "53622" ;
   dcterms:creator <http://example.com/users/steve> ;
   dcterms:created "2011-07-18T13:22:30.45-05:00" .

<http://example.com/bugs/2314/attachments/2>
   dcterms:title "fix.patch" ;
   dcterms:identifier "2" ;
   dcterms:format <http://purl.org/NET/mediatypes/text/x-diff> ;
   oslc_cm:contentSize "9196" ;
   dcterms:creator <http://example.com/users/dave> ;
   dcterms:created "2011-07-19T15:03:54.00-05:00" .

Request #3

GET http://example.com/bugs/2314/attachments/1
Accept: */*

Response #3

HTTP/1.1 200 OK
Content-Name: screenshot.png
Content-Type: image/png
Content-Length: 53622
ETag: "_87e52ce295556"

[binary content]

Create an Attachment

Once the attachment list URI for a change request is known, this is accomplished with a single POST request.

Request

POST http://example.com/bugs/2314/attachments
Content-Name: design.odt
Content-Type: application/vnd.oasis.opendocument.text
Content-Length: 18124

[binary content]

Response

HTTP/1.1 201 CREATED
Location: http://example.com/bugs/2314/attachments/3
Content-Length: 0

Alternative Approaches

Several other approaches were considered during the evolution of this draft. Following is a summary of these alternatives, along with the drawbacks of each. Note: these ideas are interrelated; they are not all mutually exclusive.

Intermediate Resource for Attachment Metadata

Initially, an attachment was actually conceived as an RDF resource describing the attachment. Such a resource would use all of the metadata properties described above, along with oslc_cm:content to reference the actual attachment content, which would be available as another resource.

This approach would complicate attachment creation, since two separate resources would need to be transmitted to the CM provider and somehow associated. Moreover, many CM systems actually require the attachment content and metadata at the same time, so the typical RESTful pattern (POST the content, obtain the resulting resource's URI, POST the metadata including the content URI) would not be possible. As a result, a single POST using multipart/mixed content was considered. This would seem a considerable complication compared to the current proposal, especially given the amount of overlap between the properties of the intermediate metadata resource and the standard HTTP headers.

Embedded Attachment Content

With the original model of an attachment as an RDF resource, an alternative to having a separate content resource would be to embed the content in the RDF resource as the value of an oslc_cm:content statement (base64 encoding was proposed for binary content). This approach would eliminate the above drawbacks, but seems particularly unRESTful. Moreover, it would complicate the process of retrieving the attachment content, requiring an RDF resource to be parsed and the content to be extracted and possibly decoded, and would be ill-suited to streaming.

No AttachmentList

Rather than having an AttachmentList resource, the attachments would be referenced directly via oslc_cm:attachment statements in the ChangeRequest. This approach would require an external factory to create attachments, which would introduce the problem of how to link attachments to change requests. One proposal was to require the POSTed attachment (originally conceived as an RDF resource containing attachment metadata) to specify the associated change request, but there was concern that such a restriction would seem arbitrary and non-obvious to consumers. In particular, it treats the change request as subordinate to the attachment when, in most CM systems, the reality is the other way around.

Next Steps

1. Identify scenarios (done)
2. Define conceptual model (done)
3. Expand on samples to support scenario (almost done)
4. Explore alternative approaches (in progress)
5. Explore any existing standards / vocabularies (in progress)
6. Finalize specification contributions