FHIR Chat · Why resource id SHALL be ignored on POST? · implementers

Stream: implementers

Topic: Why resource id SHALL be ignored on POST?

Andrew Tropin (Sep 04 2018 at 10:30):

According to fhir spec resource id SHALL be ignored on create operation, but
the reason behind this decision isn't clear for me.

I see 2 more intuitive approaches:

• Return 422, because id will be generated by the server anyway and not needed
  to be presented in the POST request body.

• Create a resource with specific id or reject with 409/422 if resource with
  such id already exists, like insert with primary key in relational databases.

I have one idea, why this decision was made, but not sure about it.

Can some one explain this part of the fhir spec please?

Grahame Grieve (Sep 04 2018 at 11:16):

we used to say that the id was prohibited, since it would be ignored

Grahame Grieve (Sep 04 2018 at 11:18):

but it turned out that people found it tiresome to remove the id from the resource before they posted it to the server

Andrew Tropin (Sep 04 2018 at 12:11):

Thanks for the answer. The reason is clear now. Actually I came to this
question from more complicated one. (There is some information on relevant
topic, but it still not clear to me how to handle the situation below).

For example, I want to create two resources inside transaction: Patient and
ProcedureRequest for this patient. I don't know patient id when I creating a
bundle. How to set a subject for procedure request in this case?

Simone Heckmann (Sep 04 2018 at 13:37):

You will need to create a uuid for each entry in your bundle and put it into Bundle.entry.fullUrl
You can use that temporary url to reference between entries in a Bundle. The server will replace UUIDs with the actual URLs when processing the Bundle

Simone Heckmann (Sep 04 2018 at 13:39):

http://build.fhir.org/http.html#trules
http://build.fhir.org/bundle.html#bundle-unique

Simone Heckmann (Sep 04 2018 at 13:40):

Here's an example: http://build.fhir.org/bundle-references.xml.html

Simone Heckmann (Sep 04 2018 at 13:42):

<entry>
    <fullUrl value="urn:uuid:04121321-4af5-424c-a0e1-ed3aab1c349d"/>
    <resource>
      <Patient>... </Patient>
    </resource>
  </entry>
...
 <!--    reference to a locally identified resource    -->
  <entry>
    <fullUrl value="http://example.org/fhir/Observation/12"/>
    <resource>
      <Observation>
       ...
        <subject>
          <!--    reference to the  patient above    -->
          <reference value="urn:uuid:04121321-4af5-424c-a0e1-ed3aab1c349d"/>
        </subject>
      </Observation>
    </resource>
  </entry>

Andrew Tropin (Oct 08 2018 at 15:00):

You will need to create a uuid for each entry in your bundle and put it into Bundle.entry.fullUrl
You can use that temporary url to reference between entries in a Bundle. The server will replace UUIDs with the actual URLs when processing the Bundle

Thanks a lot, it helped. Implemented bundle processing with dynamic resolve of references.

Andrew Tropin (Oct 08 2018 at 15:10):

we used to say that the id was prohibited, since it would be ignored

Returning to this question, why id was prohibited. It seems pretty reasonable to have ability to POST a resource with specific id, for example when we need to keep same id in legacy system and in fhir-based system. Also, in databases insert operation doesn't ignore any fields and for developer it's not an intuitive behavior of create operation. Actually, I want to deeper understand all the work behind this decision before suggesting any changes. @Grahame Grieve I would be very grateful if you explain it in more details.

Michele Mottini (Oct 08 2018 at 15:13):

To create a resource with a specified ID you use PUT, not POST (http://hl7.org/fhir/http.html#update)

Michele Mottini (Oct 08 2018 at 15:13):

POST is for 'create a new resource generating a new ID'

Andrew Tropin (Oct 08 2018 at 15:23):

To create a resource with a specified ID you use PUT, not POST (http://hl7.org/fhir/http.html#update)

Thank you for your reply. I already found possibility of upsert operation using PUT method, but it doesn't answer my question about reasons behind the decisions of ignoring id for POST method.
To be precise the question is not "How to create a resource with specific id?", but "Why I can't create a resource with specific id using POST method?".
I'm curious about why such decision was made. I think it's important question not only for fhir server implementers, but also for fhir users.

Yunwei Wang (Oct 08 2018 at 16:05):

Because id is the unique logical identifier on server. When client create a resource, client does not know if the id on client side is unique on the server side. So the client can provide business identifier but should not provide logical identifier

Lloyd McKenzie (Oct 08 2018 at 16:08):

HTTP POST and PUT are defined in the HTTP spec to have distinct behavior. We're just being consistent with that.

Andrew Tropin (Oct 08 2018 at 16:15):

Because id is the unique logical identifier on server. When client create a resource, client does not know if the id on client side is unique on the server side. So the client can provide business identifier but should not provide logical identifier

If the id isn't unique it's not a problem, server can respond with 409 for example.
Anyway it's possible to provide logical identifier using PUT method.

Andrew Tropin (Oct 08 2018 at 16:19):

HTTP POST and PUT are defined in the HTTP spec to have distinct behavior. We're just being consistent with that.

I thought about it and read HTTP rfc, PUT is implemented good according to rfc, but for POST it's not so clear.

According to rfc:

POST is designed to allow a uniform method to cover the following functions:
- Extending a database through an append operation.

But fhir create operation looks not identical to database create operation (fhir ignores id, when database not).

Thank you for your replies, but the question is still open.

Yunwei Wang (Oct 08 2018 at 16:25):

Returning 409 is not practical. The client still doesn't know what id value could be accepted by server. Then it fells into a try and fail loop.

Andrew Tropin (Oct 08 2018 at 16:45):

Returning 409 is not practical. The client still doesn't know what id value could be accepted by server. Then it fells into a try and fail loop.

Practical is a subjective term : ) If client know "the rules", it will use "proper" id (409 will make it clear that resource already created/synced/whatever), but if client don't know "the rules" or just doesn't care about id, it will not add id to resource and resource will be created with id generated by server.

Also, you are right that there are many ways to implement different use case using business identifiers, conditional creates and so on.
I try to understand a root of idea to ignore id on POST, but allow to PUT /Resource/id to create a resource with user defined id. Also, such behavior of create operation doesn't allow to create a resource with user defined id with conditional update operation.

Michael Riley (Oct 08 2018 at 17:01):

Returning HTTP 409 Conflicted is a partial security risk as you shouldn't be leaking whether resources with given ids exist if the user hasn't been been previously authorized to read the resource. It's probably not very realistic that a user could POST a resource without having at least partial read access to the same resource, but it's conceivable that other security measures would prevent certain resources from being read, and this could be a way to circumvent that obfuscation.

Andrew Tropin (Oct 08 2018 at 17:57):

Returning HTTP 409 Conflicted is a partial security risk as you shouldn't be leaking whether resources with given ids exist if the user hasn't been been previously authorized to read the resource. It's probably not very realistic that a user could POST a resource without having at least partial read access to the same resource, but it's conceivable that other security measures would prevent certain resources from being read, and this could be a way to circumvent that obfuscation.

Nice idea, but you are right it's not very realistic, also, security issues should be covered by implementers of a specific system, not by fhir spec.

Brian Reinhold (Oct 08 2018 at 18:31):

we used to say that the id was prohibited, since it would be ignored

Returning to this question, why id was prohibited. It seems pretty reasonable to have ability to POST a resource with specific id, for example when we need to keep same id in legacy system and in fhir-based system. Also, in databases insert operation doesn't ignore any fields and for developer it's not an intuitive behavior of create operation. Actually, I want to deeper understand all the work behind this decision before suggesting any changes. @Grahame Grieve I would be very grateful if you explain it in more details.

@Andrew Tropin My understanding is if you need to define the id you do an update (PUT). If the resource exists, it gets updated. If it does not, it gets created with the specified logical id. Now I see someone else already said this!

Grahame Grieve (Oct 08 2018 at 18:33):

I think the answer is pretty simple: people hate it when we define more than one way to do the same thing. So we do it as little as possible. And since we defined upsert per PUT, we don't allow it per post, specially since we found that people often leave the id in place by mistake. The PUT, you either don't include the id, or it must match, which catches the 'by mistake' cause

Andrew Tropin (Oct 09 2018 at 12:49):

I think the answer is pretty simple: people hate it when we define more than one way to do the same thing. So we do it as little as possible. And since we defined upsert per PUT, we don't allow it per post, specially since we found that people often leave the id in place by mistake. The PUT, you either don't include the id, or it must match, which catches the 'by mistake' cause

Good, looks like an answer aligned with topic of my question. Thank you for attention to wording.

I'm not sure about what duplication of functionality are you talking, that is why to be clear: I'm not suggesting to make it possible to do upsert via POST. POST for create and PUT for upsert. I try to understand a little inconsistent in behavior of those operations.

upsert (PUT /ResourceType/id) allows to create resources with custom id, conditional upsert (PUT /ResourceType?query=here) and create(POST /ResourceType) doesn't. Lack of such capabilities reduces the power of fhir transactions and makes fhir operations behavior inconsistent.

Is a fact that some people make mistakes posting resources with id included a strong argument to make decision for specification?
Just an idea, 4xx on POST with id or some explicit query param/header can solve the problem of such mistakes.

I'm asking those questions because our experience (different teams on few different projects) tells that it's more intuitive for developers to have ability to do create with custom id, rather than upsert. Therefore our fhir server have an internal opinionated POST implementation, but it is hard to keep fhir and internal POST in sync. I try to explore decision origins and probably will make a resoned proposal for next fhir version.

Patrick Werner (Oct 09 2018 at 13:11):

upsert (PUT /ResourceType/id) allows to create resources with custom id, conditional upsert (PUT /ResourceType?query=here) and create(POST /ResourceType) doesn't. Lack of such capabilities reduces the power of fhir transactions and makes fhir operations behavior inconsistent.

i agree on upsert and conditional upsert should behave similar, there was already a discussion on that topic:
https://chat.fhir.org/#narrow/stream/4-implementers/subject/Transaction.20conditional.20create.20OR.20update
and a resulting Tracker: https://gforge.hl7.org/gf/project/fhir/tracker/?action=TrackerItemEdit&tracker_item_id=17906
This tracker is already balloted, it will be part of the FHIR Build soon.

Grahame Grieve (Oct 09 2018 at 16:37):

I'm not suggesting to make it possible to do upsert via POST.

I'm not following how POST with specified Id is different to PUT with specified Id

Grahame Grieve (Oct 09 2018 at 16:37):

(I mean, in outcome)

Andrew Tropin (Oct 10 2018 at 10:34):

@Grahame Grieve, answering your question about outcome: familiar and intuitive behavior of operations for fhir user, simpler documentation, less edge cases and simpler implementation for fhir server/service developers.

In more details:

1. As a fhir user (especially newcommer) I prefer to be already familiar with behavior of create/update/upsert/conditional update operations rather then exploring tons of edge cases. Most of fhir users are developers and therefore they already familiar with classical relational databases and expects to have similar behavior of create/upsert operation (fhir doesn't have an update operation). This is my observation based on a dozen of projects and it can be not representative.

2. As a fhir server developer I prefer to keep my implementation clean and without tons of special cases, that is why it is nice to have ability to delegate create part of upsert/conditional upsert operations to same code, which performs create operation.
Different behavior of create part for create/upsert and even conditional upsert operations makes a lot of implementation problems.

@Patrick Werner I want to go deeper and suggest to redefine operations a little.

• First of all to rename update and conditional to upsert and conditional upsert not to confuse people and reduce amount of unnecessary documentation. (optional, but I think very reasonable).
• Allow to define custom id on POST /ResourceType
• Make update (actually upsert) operations use create logic (and re-use code as consequence) in case if no resource found/exist on update interaction.

As a consequence we don't need to distinguish such cases as conditional update with id should perform Update As Create and conditional update without id specific should perform create. It is just one more example, not the only benefit : )

"The key to performance is elegance, not battalions of special cases. -Jon Bentley and Doug McIlroy"

I will describe a proposal in more details, covering all cases, but I'm not sure when I have to create it, https://gforge.hl7.org/gf/project/fhir/tracker ?

Last updated: Apr 12 2022 at 19:14 UTC