FHIR Chat · Searching on extentions · implementers

Stream: implementers

Topic: Searching on extentions

Josh Mandel (Jun 23 2016 at 11:15):

Is there something we should consider doing to define a standard pattern that exposes extensions to search? (Just thinking aloud here - e.g. ?extension-string={{uri}}$value) The implication wouldn't necessarily be that servers should index every extension, but it could be nice to have some common pattern for when such indexing was desirable.)

Grahame Grieve (Jun 23 2016 at 11:18):

i see several problems with that. we've otherwise followed the map/reduce approach, and there's parameter types and modifiers. Introducing thie new approach is going to challenge these things

Grahame Grieve (Jun 23 2016 at 11:19):

and it helps how?

Josh Mandel (Jun 23 2016 at 11:26):

Its helps for reference servers and data warehouses that want to offer powerful search across lots of FHIR data that may not aways be well specified ahead of time. That's a very practical thing, to say "here's a store where you can write anything you want, and have it available and sharable and searchable in convenient ways." yes, there are parameter types, but these are explicit in the extensions value[x], so I don't see a hard constraint there. I'd be most worried about whether the syntax could be made useable.

Grahame Grieve (Jun 23 2016 at 11:37):

but if they are exposing the extensions, they can expose them with a name using the existing infrastructure

Josh Mandel (Jun 23 2016 at 11:39):

But the server might never have seen these extensions until a client posts them. So a convention would be useful (I'm not sure I see how to make this work without a convention -- even if the convention itself is not standardized).

Josh Mandel (Jun 23 2016 at 11:40):

It's similar to how clients can add arbitrary tags and make their resources searchable via standard FHIR. Only, tags have very limited semantics (no dates, or codes for example).

Brian Postlethwaite (Jun 23 2016 at 11:59):

I would suggest that you would at least want a profile with the extensions defined in them, and use the names from these.

Brian Postlethwaite (Jun 23 2016 at 12:00):

Grahame's question on how to know which type of search parameter processing to use if not defined.

Brian Postlethwaite (Jun 23 2016 at 12:02):

Do like the idea though

Josh Mandel (Jun 23 2016 at 12:02):

It would have to be part of the query syntax (like "extension-string" or "extension-dateTime").

Josh Mandel (Jun 23 2016 at 12:02):

That was the concept anyway.

Josh Mandel (Jun 23 2016 at 12:03):

Using names from a profile: which name is guaranteed to be unique (other than the url)?

Brian Postlethwaite (Jun 23 2016 at 12:04):

The one in the structuredefinition of the profile.

Josh Mandel (Jun 23 2016 at 12:04):

But, I mean globally unique?

Brian Postlethwaite (Jun 23 2016 at 12:04):

(means you have to use a profile to define searchable content, but that seems to make sense to me)

Josh Mandel (Jun 23 2016 at 12:05):

But for a generic server to work, it couldn't just take names from StructureDefinition right? Because they could clash across 2 StructureDefinitions?

Brian Postlethwaite (Jun 23 2016 at 12:06):

Hence my comment that its a single profile for the searchable content (and add in any extensions that you want to support searching with)

Josh Mandel (Jun 23 2016 at 12:07):

But I'm not following: different clients will post different resources with different extensions, and new ones all the time. Where would the server's "single profile" come from?

Brian Postlethwaite (Jun 23 2016 at 12:08):

And these would need to be in the searching profile.

Brian Postlethwaite (Jun 23 2016 at 12:08):

It's just an idea off the cuff, i'm sure there are issues with it too.

Josh Mandel (Jun 23 2016 at 12:09):

Yeah, I think the conflict resolution is a challenge there.

Brian Postlethwaite (Jun 23 2016 at 12:10):

The search profile would need to be manually created covering all the extensions on the server, and extensions that hadn't been seen wouldnt be available without updating the search profile.
Yes, these are the holes, but its just an idea.

Brian Postlethwaite (Jun 23 2016 at 12:10):

Not sure what you mean by conflict resolution there.

Grahame Grieve (Jun 23 2016 at 12:11):

I think the real question Josh has is, the servers will all name extension search parameters differently.

Grahame Grieve (Jun 23 2016 at 12:11):

too bad for clients. really.

Josh Mandel (Jun 23 2016 at 12:11):

Manual curation to resolve the case where two extensions want to use the same "name" - means that clients don't have a procedure to follow. And the "right" queries differs from server to server.

Josh Mandel (Jun 23 2016 at 12:11):

Yes, what @Grahame Grieve said :)

Grahame Grieve (Jun 23 2016 at 12:12):

we could denormalise this by putting a search name on the extension definition

Josh Mandel (Jun 23 2016 at 12:12):

But unless the search name was globally unique, we still have conflicts, no?

Grahame Grieve (Jun 23 2016 at 12:13):

that doesn't solve all a client's problems, but there is a facility in the conformance statement for remapping search names. So you could say, use the extension defined name, but we recommend the client consults the conformance statement for certainty

Grahame Grieve (Jun 23 2016 at 12:13):

or you can just say, if you want this, the clients haev to read the conformance statement, and you can always know because you will find the extension url in the conformane statement

Grahame Grieve (Jun 23 2016 at 12:14):

that would be much more consistent with our intended approach

Josh Mandel (Jun 23 2016 at 12:14):

That's true. Can you help us see why it would better the extension url as a "search name"?

Grahame Grieve (Jun 23 2016 at 12:15):

specifically, that means that you look at the search parameters, and if Conformance.rest.resource.searchParam.definition is the url, then you know the search parameter name to use

Grahame Grieve (Jun 23 2016 at 12:15):

not sure what you're asking there

Brian Postlethwaite (Jun 23 2016 at 12:16):

Would we include something in the conformance statement that said, we support searching any property, including extensions.

Brian Postlethwaite (Jun 23 2016 at 12:16):

And should that go into the filter section?

Josh Mandel (Jun 23 2016 at 12:17):

The procedure you're describing makes senses and it works. It also relies on a level of indirection (asking the server to map extension URLs to query parameter names) that could be avoided by using a convention. I'm just probing that choice.

Grahame Grieve (Jun 23 2016 at 12:17):

uh? but you can't support 'everything'. Want to take a fluent path search?

Brian Postlethwaite (Jun 23 2016 at 12:17):

Is that something that CQL wants to do too?

Grahame Grieve (Jun 23 2016 at 12:18):

no. and please no

Josh Mandel (Jun 23 2016 at 12:18):

Well, "fluent path search" wouldn't be indexable (because productions are unbounded).

Josh Mandel (Jun 23 2016 at 12:18):

Searching based on (all) extentions could be indexed and made performant in some data warehouse environmentally at least.

Grahame Grieve (Jun 23 2016 at 12:19):

@Josh Mandel : well, it makes for simpler syntax in the search, which is a known problem already. and it leverages the existing system - and clients are already supposed to check the conformance statement. (though size is a known problem). And it means the servers can continue to use the map/reduce based system which has real advantages. and it means that the type of the search parameter is known, and there's no problems with guessing the search name, or imposing glbal unqieuness, which is a problem we've explored before

Grahame Grieve (Jun 23 2016 at 12:20):

I'm not sure that searching based on all extensions is a particularly boundable problem either. but the scheme I'm outlining does make it work<, I think

Josh Mandel (Jun 23 2016 at 12:20):

I can't follow what you mean about map/reduce. Why would a naming convention change the viability of your indexing strategy ?

Grahame Grieve (Jun 23 2016 at 12:22):

not so much the naming converntion as boudning the problem, saying, 'these are the names we search on'. Using auto-genreated names makes it unclear whether it's a bounded problem. If you think of it as a bounded problem, only with controlled names, then that's not a factor. but that wasn't obvious to me

Josh Mandel (Jun 23 2016 at 12:22):

It's bounded by the size of your resource data!

Grahame Grieve (Jun 23 2016 at 12:23):

so you do it on the fly? you better have a good naming system. So much fun. I prefer just declaring the assigned name in the conformance resource

Josh Mandel (Jun 23 2016 at 12:23):

When you say "good naming system" I'm still trying to understand why extension URLs aren't "good names" in this sense.

Brian Postlethwaite (Jun 23 2016 at 12:24):

(On teh size of the conformance statement, did you see the tracker I looged suggesting that we should be able to filter the conformance statement to reduce its size, just to a specific resource type)

Grahame Grieve (Jun 23 2016 at 12:24):

the full url? they're great names, but we now khow well people amange syntax in URLs. also, extensions may not have fixsed types

Grahame Grieve (Jun 23 2016 at 12:25):

yes I did.

Josh Mandel (Jun 23 2016 at 12:26):

Extension Values have fixed types in instance data though and searches could be tied to those (that is, you'd writer a search to find string values, or dates values) . Re URLs in URLs, yes it's a pain but we already live this pain every time we search by token types (embedding codings as search parameters).

Grahame Grieve (Jun 23 2016 at 12:27):

so you're only going to solve a subset of this problem, then, only for when the data is already known. And as the data grows, the searches get altered?

Grahame Grieve (Jun 23 2016 at 12:27):

sounds like a problem to me

Josh Mandel (Jun 23 2016 at 12:28):

I wasn't suggesting searches would 'get akterrd". Or that those is a subset of the problem.

Josh Mandel (Jun 23 2016 at 12:29):

Today when we have choice types in fhir, we have multiple search parameters (one per type). Like Observation value-xxx. The thing I'm describing with extensions is totally equivalent.

Grahame Grieve (Jun 23 2016 at 12:29):

but it seems to me that you are making assumptions that mean it will be a subset of the problem

Grahame Grieve (Jun 23 2016 at 12:29):

e.g. the server knows how to search because the of the data it already has. Ergo, new data revise the search method

Josh Mandel (Jun 23 2016 at 12:30):

I'm not seeing how the method is "revised" by new data. The method is always "index each incoming resource based on the extension value[x]s found inside". You'd never go back and revise or change results, because they'd remain correct until a resource's content changed.

Grahame Grieve (Jun 23 2016 at 12:37):

hmm I misread you, I think. so you're thinking about

GET Observation?code=XXX&http://acme.org/fhir/StructureDefinition/my-long-extension-name-string=yyyy

Brian Postlethwaite (Jun 23 2016 at 12:39):

How would I know which search type to use on the item? a string property could be a token or string

Grahame Grieve (Jun 23 2016 at 12:40):

that's the -string at the end. it doesn't really stand out does it?

Brian Postlethwaite (Jun 23 2016 at 12:41):

ok, I think I get it too.

Grahame Grieve (Jun 23 2016 at 12:45):

I guess I could implement that for some extensions (I wouldn't necessarily do all of them). I'd like a syntax that makes the type stand out rather than using a character that is part of a valid resoruce id

Josh Mandel (Jun 23 2016 at 14:47):

Yes that's the basic idea I was trying to get across. Agreed, making the type stand out (and parseable separate from the extension URL, unambiguously) would clearly help :-)

Eric Haas (Jun 23 2016 at 17:40):

OK for the rest of us ...can you point out in your generic notation and example where the type would be?

Eric Haas (Jun 23 2016 at 17:41):

(grahame's example was cut off too )

Josh Mandel (Jun 23 2016 at 18:17):

Sure @Eric Haas ! I was thinking, imagine you have a resource like:

{
  "resourceType": "Patient",
  "extension": [{
    "url": "http://example.com/astrological-sign",
    "valueCoding": {
      "system": "http://example.com/astrology/signs",
      "code": "pisces"
    }
  }, {
    "url": "http://example.com/favorite-word",
    "valueString": "supercalifragilisticexpialidocious"
  }]
}

Then you could write queries like (and don't take my syntax too literally, please):

GET /Patient?_extension=http://example.com/astrological-sign$coding$pisces
GET /Patient?_extension=http://example.com/favorite-word$string:contains$cali

Grahame Grieve (Jun 23 2016 at 21:15):

I would have thought that this would be better:

GET /Patient?http://example.com/astrological-sign$coding=pisces

Eric Haas (Jun 23 2016 at 21:20):

(deleted)

Eric Haas (Jun 23 2016 at 21:20):

nm

Chris Grenz (Jun 23 2016 at 21:25):

IMO - using the existing SearchParameter infrastructure and allowing clients to easily query for the extension they're after would be preferable...

Josh Mandel (Jun 23 2016 at 21:28):

Yes that's nice! I said not to take my syntax too seriously :)

Josh Mandel (Jun 23 2016 at 21:31):

Could be, @Chris Grenz! But I'm not sure the "easy" aspect is what is most appealing about the two step process in our current infrastructure :-)

Chris Grenz (Jun 23 2016 at 21:32):

I'm thinking about access control...we currently secure by SearchParameter...this is unbounded

Chris Grenz (Jun 23 2016 at 21:32):

Also indexing, but that could be handled...

Chris Grenz (Jun 23 2016 at 21:33):

Might be better for a server that wants wide flex on this to just create new SearchParameters whenever a new extension is encountered....

Grahame Grieve (Jun 23 2016 at 21:33):

not necessarily. You'd still actually need to support this with a SearchParameter. In essence, this is just an approach to fixing the name that the server declares in it's search parameter in order to allow a client to skip the reflection step if it wants

Chris Grenz (Jun 23 2016 at 21:34):

@Josh Mandel were you expecting that?

Josh Mandel (Jun 23 2016 at 21:50):

Yes, I was expecting that servers *could* still expose this information in Metadata.

Chris Grenz (Jun 23 2016 at 21:50):

I think Grahame is saying *must*.

Josh Mandel (Jun 23 2016 at 21:51):

I mean, that wouldn't bother me, if clients aren't required to invoked it :p

Chris Grenz (Jun 23 2016 at 21:52):

Fair enough...so the server (or some actor) would have to create these as extensions are encountered...

Chris Grenz (Jun 23 2016 at 21:52):

Could do this today

Josh Mandel (Jun 23 2016 at 21:53):

As in "no deep architectural impediments", or "less than 4 hours of work"? ;)

Chris Grenz (Jun 23 2016 at 21:54):

Yes?

Josh Mandel (Jun 23 2016 at 21:54):

:p

Chris Grenz (Jun 23 2016 at 21:55):

It's not going to be terribly portable though

Grahame Grieve (Jun 23 2016 at 23:22):

I didn't quite say 'must', Servers are allowed to support search parameters they don't declare.

Brian Postlethwaite (Jun 24 2016 at 01:33):

I've had a chance to think overnight on this, and I don't like it anymore.
Its trying to say that here is a naming convention for search parameters that is excplicit for extensions.
Why do we not have the same thing for regular fields too?

GET /Patient?animal.genderStatus=intact

Brian Postlethwaite (Jun 24 2016 at 01:34):

This of course clashes with the chained search parameters, but I guess so do the URLs with dots in them.

Josh Mandel (Jun 24 2016 at 01:55):

Regular search params don't always point to a single path (they can be Observation.code | Observation.component.code for example), so the simple rule wouldn't quite work. That said, @Brian Postlethwaite: I would rather see some conventions on how regular search params were named, too (but I think that ship has sailed).

Brian Postlethwaite (Jun 24 2016 at 02:24):

Agreed.

Brian Postlethwaite (Jun 24 2016 at 02:24):

Then again, if you're defining an extension, maybe you should be defining a search name to use also.

Brian Postlethwaite (Jun 24 2016 at 02:25):

Then the registry would not only have the extension definition, but the search definition too.
Would that assist in resolving the issue?

Lloyd McKenzie (Jun 24 2016 at 03:16):

Challenge when searching by extension is that extensions can appear at different places in an instance - and you might want to search on those that appear in one location but not another (even though they'll have the same URI.

Josh Mandel (Jun 24 2016 at 03:17):

You might!

Grahame Grieve (Jun 24 2016 at 03:18):

really? is that important?

Josh Mandel (Jun 24 2016 at 03:22):

I wasn't saying I thought it was important. Just possible. I'd be fine with context-free, or nearly.

Last updated: Apr 12 2022 at 19:14 UTC