FHIR Chat · The new, package-based registry.fhir.org · tooling

Stream: tooling

Topic: The new, package-based registry.fhir.org

Ward Weistra (Sep 10 2020 at 14:42):

Hi all!

The :new: registry.fhir.org, recently demonstrated to HL7 members, is live! :tada:
The FHIR Registry is now FHIR package-based, only including contents published as a package via one of the package feeds (e.g. via HL7, FHIR Foundation or Simplifier.net).

The registry can be used to:

• Find your Implementation Guide / Package of interest ("I want to install the Structured Data Capture FHIR package, or use or view the Implementation Guide", "I want to see the Blue Button FHIR resources")
• Find specific resources to build on or be inspired by ("Are there any blood pressure profiles in FHIR R4 I can use?", "Are there any COVID-19 ValueSets or Questionnaires available? ", "Is there any work done on Genomics, Dental Care, Breast Cancer?")

Search can be done by free text, filters, but also by entering canonicals of packages/Implementation Guides or specific resources.
The registry.fhir.org interface is meant for humans, computers can talk to the package endpoint documented here: https://simplifier.net/docs/package-server

Please share your questions and feedback here! If you're quick we might be able to resolve some of them within the current project :smile:
This project was funded by HL7 International (cc @Grahame Grieve @Wayne Kubick) and built by Firely (cc @Brian Kaney @Vicki Strateman Dewar @Martijn Harthoorn @Paul Deac).

May Terry (Sep 10 2020 at 14:47):

Thanks for posting! Is there any top-level navigation that allows for browsing? One nice thing about simplifier.net was the ability to see a splash page of profiles by resource: https://simplifier.net/resources. I don't see one in registry.fhir.org.

Ward Weistra (Sep 10 2020 at 14:55):

Some known issues, previous feedback:

• Pages Privacy, Contact, Credits haven't been created yet
• Put an (i) information icon next to NPM/Torinox command on package overview page. Link to right section of explanation on using FHIR packages?
• Latest releases only' might not be clear as a toggle
• Page titles seem to be URLs

Search tuning examples:

• http://34.120.254.211/results?query=%22us+core+patient%22&latestFilter=true Gives US Core (and its patient profile) only end of first page. (via @Josh Mandel)
• While the top results are always relevant and unique, somehow the same low-quality packages are showing up on the bottom of the list. Are we accidentally benefitting them? (empty fields?)

Ward Weistra (Sep 10 2020 at 14:58):

May Terry said:

Thanks for posting! Is there any top-level navigation that allows for browsing? One nice thing about simplifier.net was the ability to see a splash page of profiles by resource: https://simplifier.net/resources. I don't see one in registry.fhir.org.

Good question, May. The focus for the registry is currently to find a package or resource of interest. You can do an empty search (https://registry.fhir.org/results?query=%22%22&latestFilter=true) to see all packages, but it won't be very useful.
The Simplifier resource overview will continue to exist, maybe someday even with a filter specifically for package content only.

Paul Denning (Sep 11 2020 at 13:04):

Why isn't the new version of US Core showing up http://hl7.org/fhir/us/core/STU3.1.1/ ?

Ward Weistra (Sep 11 2020 at 13:28):

@Paul Denning Great observation! I see it came in three days ago (https://simplifier.net/packages/hl7.fhir.us.core) and you've correctly noted that filling the index is still in the process of being automated. This will run at least once a day very soon.

Ward Weistra (Sep 24 2020 at 23:37):

Update on the Registry

Blog post published with introduction to the Registry: https://blog.fire.ly/2020/09/24/the-new-fhir-package-registry/

Updates to the registry since last time:

• Added Google Analytics to stear future product decisions
• Tuned search to remove a set of low-quality packages that came up in every search
• Jurisdiction: Recently a change to the package spec was approved to include jurisdiction in the package manifest. We've added the UI components for this, which can be turned on ones the data comes in.
• Added a filtering option on the Contents tab of the package overview page. When you click on a resource in the search results it will use this to take you straight to the resource.
• Added an explanation when a package is in the search results because of the package metadata only
• Added an information link to the package install widget
• Added human readable page titles on every page
• Clarified the toggle for searching only in latest packages or not

John Moehrke (Jun 01 2021 at 18:41):

@Ward Weistra how can I hook IHE publication of IG packages into the new registry.fhir.org?

John Moehrke (Jun 01 2021 at 18:41):

I was publishing to the ig-registry github repo, but my pull-request continues to be outstanding.

Grahame Grieve (Jun 02 2021 at 00:04):

you need a feed, and the feed to be registered in ig-reigstry

Grahame Grieve (Jun 02 2021 at 00:05):

and I just merged it.

Ward Weistra (Jun 02 2021 at 09:45):

@John Moehrke Indeed, what Grahame says. @David Hay also wrote it out here: https://confluence.hl7.org/display/FHIR/FHIR+Package+Registry+User+Documentation#FHIRPackageRegistryUserDocumentation-Themanualprocess
I don't believe the PR Grahame merged includes any package feeds yet, right?

Once your feed is listed in the feeds object of https://github.com/FHIR/ig-registry/blob/master/fhir-ig-list.json Simplifier.net will start importing them. Once they're in Simplifier.net they are also available via the package API endpoint. A bit later they will also be in the UI of registry.fhir.org.
The import from known feeds in the list happens a few times per day, not sure how often the new feed discovery is. Let me know if they don't show up in Simplifier.net (https://simplifier.net/packages) in a few days after the feed is listed.

Grahame Grieve (Jun 02 2021 at 10:21):

the publisher can build the feeds for you, but that requires that you use the publisher. Else, you can make your own by copying one of the others. THey're pretty simple

John Moehrke (Jun 02 2021 at 11:27):

I understand that the IG publisher is key, we are now using that. I will look at the confluence to see what we need to do. Sorry I didn't know about this.

Ward Weistra (Jun 03 2021 at 10:08):

@John Moehrke It's pretty new, not done often yet. Publishing with Simplifier.net or anything that generates FHIR packages + a FHIR package feed will work too.

John Moehrke (Jun 03 2021 at 11:02):

@Ward Weistra How often does the registry look to the ig-registry? Grahame accepted my pull-request a day and a half ago, yet the registry still does not show formatcode or mhd.

Ward Weistra (Jun 03 2021 at 11:27):

@John Moehrke Which feed in the feeds object of fhir-ig-list.json is IHE's then? :thinking:
As far as I can see your PR only made changes to the guides part.

Ward Weistra (Jun 03 2021 at 11:30):

You'll need a package feed like http://hl7.org/fhir/package-feed.xml for IHE and then list that one in the feeds list, like so:

    {
      "name": "HL7.org Publications",
      "url": "http://hl7.org/fhir/package-feed.xml",
      "errors": "grahame|fhir_org"
    },

John Moehrke (Jun 03 2021 at 11:40):

understood.. but I am wondering why the merged pull-request didnt result in those showing up?

John Moehrke (Jun 03 2021 at 11:43):

This was done the old way, that I understand should still work. I want see that show up so that I can tell that mechanism works for the IHE published IGs.

John Moehrke (Jun 03 2021 at 12:51):

@Ward Weistra I have added the feed xml for IHE. Can you confirm that it is properly formatted?
https://profiles.ihe.net/fhir/package-feed.xml

John Moehrke (Jun 03 2021 at 13:01):

@Grahame Grieve I think I have created the pull-request on ig-registry needed to add in the IHE rss feed

John Moehrke (Jun 03 2021 at 13:03):

I will be updating both FormatCode and MHD ... so want to make sure the mechanism is working.

Ward Weistra (Jun 03 2021 at 14:17):

John Moehrke said:

This was done the old way, that I understand should still work. I want see that show up so that I can tell that mechanism works for the IHE published IGs.

The guides object in fhir-ig-list.json is only affecting what shows on https://www.fhir.org/guides/registry/, not the package registry. Two 'separate' worlds in one JSON file.

Ward Weistra I have added the feed xml for IHE. Can you confirm that it is properly formatted?
https://profiles.ihe.net/fhir/package-feed.xml

Once it's merged into fhir-ig-list.json we can run the importer and I can let you know for sure. By eye:
<language>en></language> Probably has a > too many, although it likely won't be used yet. (same for HL7 feed though)

Re: 'updating' the packages (and this is probably superfluous) every combination of package name + version will only be imported once. You can of course add new versions to be picked up.

John Moehrke (Jun 03 2021 at 14:30):

that language bug is in the HL7 one too. I will fix mine, but not sure how to get HL7 one fixed.

John Moehrke (Jun 03 2021 at 14:31):

I was not sure what the format of <description> was... but I saw others with multiple paragraphs, so went with it.

John Moehrke (Jun 03 2021 at 14:31):

I presume the new ones are added at the top?

Ward Weistra (Jun 03 2021 at 15:17):

@John Moehrke Order shouldn't matter. Just make sure package name + version is unique is all.

Grahame Grieve (Jun 03 2021 at 22:03):

merged. now that I'm awake. And I fixed the HL7 feeds

Ward Weistra (Jun 09 2021 at 13:02):

@John Moehrke They're everywhere now: https://registry.fhir.org/results?query=%22ihe%22&latestFilter=true
Did you for ihe.mhd.fhir briefly publish a 4.0.0-comment version? https://registry.fhir.org/package/ihe.mhd.fhir%7C4.0.0-comment. That one has been picked up, the 4.0.0 version that is in the feed now has currently not been.

John Moehrke (Jun 09 2021 at 13:51):

4.0.0 was our public comment one. Final pub is 4.0.1... 4.0.1 should be the latest pub.

Ward Weistra (Jun 09 2021 at 19:03):

Great, that's what's reflected in the registry

John Moehrke (Jun 17 2021 at 17:26):

is there a way to find “all IGs that profile the X resource”?

Use-case is, I know that I want to use the CarePlan resource, but don’t know if someone else has already done the profiling that I think I need to do.

Grahame Grieve (Jun 17 2021 at 19:00):

http://clinfhir.com/igAnalysis.html

John Moehrke (Jun 17 2021 at 19:17):

not working. @David Hay ?

David Hay (Jun 17 2021 at 19:49):

I'm innocent I tell you, @Ward Weistra ?

Grahame Grieve (Jun 17 2021 at 19:50):

possibly, but can you tell me what api call is timing out?

David Hay (Jun 17 2021 at 19:50):

oops - it is me - hold on...

David Hay (Jun 17 2021 at 19:53):

CORS error to http://test.fhir.org/usage-stats

John Moehrke (Jun 17 2021 at 19:55):

I appreciate David doing this.. but I really think this should be an IG registry function. it is so very core need that everyone should use.

Grahame Grieve (Jun 18 2021 at 07:43):

try again

Ward Weistra (Jun 18 2021 at 08:43):

John Moehrke said:

I appreciate David doing this.. but I really think this should be an IG registry function. it is so very core need that everyone should use.

Hi John, is this the same functionality you described in your comment on our blog? If so, please see my response there.

A breakdown of the resources within a package is already in a rudimentary form on Simplifier.net package pages (which are linked from the Registry pages), like so: https://simplifier.net/packages/ihe.mhd.fhir/4.0.1/~files.

There's an HL7 Jira board for feedback/feature requests on the Registry too: https://jira.hl7.org/secure/RapidBoard.jspa?rapidView=48

John Moehrke (Jun 18 2021 at 12:07):

excellent. thanks.

John Moehrke (Jun 18 2021 at 16:57):

@David Hay I see that clinfhir is now showing the igAnalysis... but I am not finding my IGs in your list of IGs. Are you pulling from the ig-registry and/or the new registry.fhir.org?

David Hay (Jun 18 2021 at 19:44):

My understanding is that this app (http://clinfhir.com/igAnalysis.html) gets a feed directly from the build environment - rather than the IG registry. Graham would need to confirm the details of the source... What's the url of your IG?

BTW - I am working on a new viewer app (http://clinfhir.com/packageViewer.html) that can download from the registry (via its API) so would be good to try that out as well. The focus is different to the igAnalysis as it presents a view of the contents of a single IG rather than summaries across many...

John Moehrke (Jun 18 2021 at 20:06):

https://profiles.ihe.net/ITI/MHD/

John Moehrke (Jun 18 2021 at 20:06):

The 4.0.1 version is still waiting for pull-request merge into the ig-registry... so that might be why it is missing.

John Moehrke (Jun 18 2021 at 20:09):

I see 4.0.2, which i just pushed to ci-build today... so it does seem you are following build.fhir.org/ig ??

John Moehrke (Jun 18 2021 at 20:10):

so, not those that are not using the build.fhir.org/ig service...

John Moehrke (Jun 18 2021 at 20:11):

hmmm. MHD does have profiles on AuditEvent (as you could imagine). These are not showing up in your tab on Resource

John Moehrke (Jun 18 2021 at 20:12):

auditEvent should also show my basicAudit IG... which is in your list... moehrke.basicaudit#0.1.0-beta.json

John Moehrke (Jun 18 2021 at 20:16):

hmm, @David Pyke and my April Fools Day IG isn't listed... http://build.fhir.org/ig/DavidPyke/HotBeverage/

David Hay (Jun 19 2021 at 08:27):

@John Moehrke Here you go: http://clinfhir.com/packageViewer.html#ihe.mhd.fhir%7C4.0.1

early days of course...

Grahame Grieve (Jun 19 2021 at 20:33):

this data is based on something else, it's built by the IG publisher as it builds IGs

Ward Weistra (Jan 13 2022 at 11:10):

The registry.fhir.org UI is down at the moment:
image.png
We're looking into it and work to bring it back.

The API side of things (packages.fhir.org / packages.simplifier.net) is still up and running :glowing_star:

Ward Weistra (Jan 17 2022 at 17:06):

FYI, it was back up (with some nice improvements!) later that day.

John Moehrke (Jan 21 2022 at 22:13):

Really need far more easy way to search for IGs that
a) have Profiled X resource.
b) have depends on Y IG
c) have used valueSet or codeSystem Z

Josh Mandel (Jan 21 2022 at 22:15):

Do the relevant data exist in registry.fhir.org / npm packages? Or does a proper answer require consulting GitHub, national program websites, hl7 affiliates, etc?

Ward Weistra (Jan 21 2022 at 22:18):

Thanks for the feedback!

a) have Profiled X resource.

This is next on the list and we are working on designs for this, which I can likely share soon.

Ward Weistra (Jan 21 2022 at 22:19):

b) have depends on Y IG

Good one, so basically the reverse of the dependencies tab, right? Which packages have this package (IG) as a dependency.

Ward Weistra (Jan 21 2022 at 22:23):

c) have used valueSet or codeSystem Z

Would be similar, but then on a resource level.

Ward Weistra (Jan 21 2022 at 22:32):

a) is currently funded by HL7 (together with some maintenance). I've captured b) and c) as feature requests and will discuss them with whoever takes over for Wayne on this, which I assume to be @Daniel Vreeman .

Ward Weistra (Jan 21 2022 at 22:36):

Josh Mandel said:

Do the relevant data exist in registry.fhir.org / npm packages? Or does a proper answer require consulting GitHub, national program websites, hl7 affiliates, etc?

The package and resource dependencies, including base resources, are all in the packages. It just requires a lot more indexing and calculating by Simplifier.net (which forms the backend, as you'll likely know). Given that that part is Firely's contribution (not funded by HL7) it'll have to compete with our other feature requests...

Josh Mandel (Jan 21 2022 at 22:37):

My question is whether all of the content John cares about is in fact being routinely submitted to the registry.

Ward Weistra (Jan 21 2022 at 22:38):

By content you mean the specific IGs, not the attributes he's asking to search on?

Josh Mandel (Jan 21 2022 at 22:39):

Yes, the IGs themselves

Elliot Silver (Jan 21 2022 at 22:39):

But you don't need the IGs for those queries, do you? just the packages, right?

Josh Mandel (Jan 21 2022 at 22:40):

Whatever you want to call it -- I'm asking whether the relevant projects are represented in the registry

Elliot Silver (Jan 21 2022 at 22:41):

No, probably not all projects, maybe not even most, but it's the best we've got.

Ward Weistra (Jan 21 2022 at 22:43):

These are the feeds with publicly published packages/IGs currently being picked up.

Josh Mandel (Jan 21 2022 at 22:43):

I just don't have the context to know what fraction of real-world use is represented by these feeds. Some? Most? Nearly all?

John Moehrke (Jan 21 2022 at 22:49):

from the feed, they can pull the whole of the IG.

John Moehrke (Jan 21 2022 at 22:50):

I can say that the feed works well. I publish new IHE stuff, and within the hour I can see it fully navigable in the registry.

Ward Weistra (Jan 21 2022 at 23:01):

Josh Mandel said:

I just don't have the context to know what fraction of real-world use is represented by these feeds. Some? Most? Nearly all?

I am not aware of publicly available FHIR publications that are not in there. Everyone publishing via HL7 International, FHIR.org and Simplifier.net gets in automatically. And everything historically published with them is in there. And then there's some extra feeds from people publishing with the IG publisher in other locations like certain HL7 affiliates, IHE International.

Daniel Vreeman (Jan 21 2022 at 23:19):

Yeah, just yesterday I was looking to see whether you could determine which packages an IG depended on (and which ones depended on that one) via the Simplifier API (for registry.fhir.org) or search capabilities. These will be good to talk about.

Grahame Grieve (Jan 27 2022 at 04:30):

@Josh Mandel most real world usage by consortiums and national programs gets in there, but some european county IGs don't get into the registry (it's up to them to make it so). Vendors are not generally putting their stuff in the registry, though they could choose to do so.

All the information @John Moehrke wants is in the package file, though it's FHIR version dependent how to dig it out

John Moehrke (Jan 27 2022 at 15:55):

I don't think I want anything. To me the system is working great. I have IHE integrated into both registries.

John Moehrke (Jan 27 2022 at 15:56):

(not sure I want the April Fools IG to be in the registry -- @David Pyke do we want that?)
But this does open a question of if there is a way other than the feed for individuals and small projects to request their IG get added to the simplifier registry? Lets just say that Dave does want his April Fools IG in there, is there a process that is short of the feed?

David Pyke (Jan 27 2022 at 16:08):

I think it definitely needs to be in the registry. There's already an implementation started.

John Moehrke (Jan 27 2022 at 16:08):

does it only run on April 1st?

David Pyke (Jan 27 2022 at 16:08):

We shall see. It may only run at WGMs

John Moehrke (Jan 27 2022 at 16:10):

@Ward Weistra is there a manual way for individual IGs to be added to the registry?
@Grahame Grieve is this a legitimate use of the registry? Or do we want to have some minimal governance bar?

Ward Weistra (Jan 27 2022 at 16:59):

@John Moehrke You can create a free project on Simplifier and publish from there, although for a pre-made package from the IG publisher you will need to use a feed or create a new one.
And we try to encourage people to only publish quality content to the registry. But eventually I think we should just rely on usage metrics (downloads + others depending on it) to bring the best results to the top of the search. Just like there's many non-relevant websites on the internet.

John Moehrke (Jan 27 2022 at 17:03):

I like the social-graph concept... it would help everyone, even the formal IG publishers.

John Moehrke (Jan 27 2022 at 17:04):

would be nice if there was a way to get an IG publisher product into simplifier without the feed. not sure how needed this is.

Grahame Grieve (Jan 27 2022 at 19:17):

would be nice if there was a way to get an IG publisher product into simplifier without the feed. not sure how needed this is.

the problem is security. And I'm not motivated to invest in any solution to that. But there is a solution that works at low scale: ask me to put it in the fhir.org package feed.

Grahame Grieve (Jan 27 2022 at 19:17):

But eventually I think we should just rely on usage metrics

the API would need to cover those things then

John Moehrke (Jan 27 2022 at 19:59):

okay

Michael Lawley (Apr 01 2022 at 06:05):

Has any thought been given to supporting 'latest' as a version (as per npm). I see a dist-tags entry in the metadata that links 'latest' with a given version, but it would be really nice if https://packages.simplifier.net/[package]/latest just resolved automatically

Grahame Grieve (Apr 01 2022 at 20:35):

@Ward Weistra

Ward Weistra (Apr 04 2022 at 17:33):

@Michael Lawley Nice idea, though there's indeed a suitable workaround with one more API call. If you'd like you can make the feature request on https://jira.hl7.org/secure/RapidBoard.jspa?rapidView=48: report an issue in the HSCR project with component = FHIR Registry.

Gino Canessa (Apr 04 2022 at 17:37):

FWIW, my internal support of latest is an abomination of version + publication date of what is available. It's an abomination because not all versions are semver, and not all packages include date data.

Love the possibility of having it tagged!

Ward Weistra (Apr 04 2022 at 18:15):

@Gino Canessa As Michael alluded to, you can already find via the API which package to take as the latest via the dist-tags: https://app.swaggerhub.com/apis-docs/firely/Simplifier.net_FHIR_Package_API/1.0.1#/default/get__package_name_

Michael Lawley (Apr 05 2022 at 01:46):

Done: HSCR-431

Last updated: Apr 12 2022 at 19:14 UTC