FHIR Chat · Setting up a server to publish IGs with redirection? · IG creation

Stream: IG creation

Topic: Setting up a server to publish IGs with redirection?

Bob Milius (Jun 04 2020 at 20:07):

We're developing a couple of organizational IGs (non-HL7) that we want to host. If possible, I'd like to do this using the same tooling and methods as HL7 uses. For example, we'd like to have a CI/CD environment. Travis? Jenkins? does it matter? Also, I'd like to use the same tooling that HL7 uses for redirection of structuredefinition urls (e.g., http://hl7.org/fhir/us/core/StructureDefinition/us-core-patient gets redirected to http://hl7.org/fhir/us/core/STU3.1/StructureDefinition-us-core-patient.html). From reading this thread, I see a comment from @Grahame Grieve saying there's a publish.ini file that is used to generate code to do this depending on whether the server parameter is set to Apache or ASP. Does this need to exist before running the igpublisher? or is it something read by other tooling? And where exactly does it go? I've seen someone mentioninput/pages/publish.ini I can't seem to find any clear documentation around it. I've seen the Process for Publishing a FHIR IG and FHIR Guide to Authoring Types, but it doesn't have much of a documentation around publish.ini (just a bit around [pages] and talks about "shared types" which I admit I don't understand. Any clarification will be greatly appreciated.

Jose Costa Teixeira (Jun 04 2020 at 20:19):

@ Bob Milius I recently went through the process of manually publishing an IG - i did it manually and may have cut some corners, but it turns out to be not so complicated in the essence

Bob Milius (Jun 04 2020 at 20:21):

@Jose Costa Teixeira I've been able to manually upload an IG to an S3 bucket. I'm looking more toward the redirection bit, and adding a CI/CD environment, as well as having a process for current builds, and milestone releases for our organization. Have you been able to do that?

Jose Costa Teixeira (Jun 04 2020 at 20:21):

I'm not sure but I think the redirects are a result of the packages

Jose Costa Teixeira (Jun 04 2020 at 20:22):

CI- CD - interesting, I'm on to it as well. Your idea to use Jenkins may just help me a lot

Jose Costa Teixeira (Jun 04 2020 at 20:22):

my first step was to get the publisher to do this (notice the publish box):
https://hl7belgium.org/fhir/2020May

Bob Milius (Jun 04 2020 at 20:29):

@Jose Costa Teixeira the IG looks nice! But I'm not seeing the redirection. e.g., I think the url https://www.ehealth.fgov.be/standards/fhir/StructureDefinition/be-patient should redirect to https://hl7belgium.org/fhir/2020May/StructureDefinition-be-patient.html, shouldn't it?

Bob Milius (Jun 04 2020 at 20:31):

or [igname]/StructureDefinition/[profile] should redirect to [igname]/StructureDefinition-[profile]

Jose Costa Teixeira (Jun 04 2020 at 20:36):

i did not setup the redirects. Perhaps we can do that soon?

Jose Costa Teixeira (Jun 04 2020 at 20:36):

and indeed I have to clean up the urls - this IG actually uses resources that are from another IG, and I am not sure how best to handle it

Jose Costa Teixeira (Jun 04 2020 at 20:38):

I'll check my notes about getting the publisher to move through the release cycle (ci-build, qa-preview, final...) and I will share them

Jose Costa Teixeira (Jun 04 2020 at 20:40):

(I took those notes sometime before my computer got attacked, so i don't have them here). But I started with a simple vanilla process, and now I hope to enrich with all these details - that is the process that works for me: remove parts until I see how it works, then add those parts again

Grahame Grieve (Jun 04 2020 at 20:55):

@ Bob Milius:

• the publishing thing is a separate mode that you run the IG publisher in at a different time in the process.
• it's true that this is very poorly documented
• the redirects aren't going to work on an S3 bucket. They need active logic on the server, and there's no way I've found to do that on an S3 bucket
• Setting up a CI build if you can't use the public one is a much bigger piece of work - that's real infrastructure...

John Moehrke (Jun 04 2020 at 21:38):

so, what is required? I am trying to get IHE publication site and need to give system requirements.

Grahame Grieve (Jun 04 2020 at 21:51):

you need a web server. Currently, we support Apache, Nginx, Lightspeed, and IIS

Grahame Grieve (Jun 04 2020 at 21:52):

and technically, the sub-directory that hosts the FHIR content needs to be able to run ASP or PHP scripts. I can provide more specific requirements by web server

Bob Milius (Jun 04 2020 at 22:23):

Grahame Grieve said:

Bob Milius:

• the redirects aren't going to work on an S3 bucket. They need active logic on the server, and there's no way I've found to do that on an S3 bucket

The S3 thing was just to quickly share. We are planning on deploying a dockerized Apache server. Any advice on this would be welcome

• Setting up a CI build if you can't use the public one is a much bigger piece of work - that's real infrastructure...

I'm assuming we can't use the public one for a non-HL7 IG. Is that true?
We have devops guys who can set up the infrastructure. I was just wondering if there are preferred public tools/workflows we can reuse. They are talking Atlassian Bamboo for CI/CD. Maybe that would be easiest for us.

Chris Moesel (Jun 04 2020 at 22:23):

According to this S3 documentation, you can setup redirects with Amazon S3. I've never tried it before though.

I just experimented with a poor man's approach that seems to work, assuming you're hosting your IG at a URL matching its canonical root. It takes advantage of two things:

1. Most web servers are setup such that if you request a path matching a folder, it will serve the index.html from that folder
2. The META refresh directive

Basically, if my IG has canonical URL: http://example.org/fhir/myIG/ and is also hosted at http://example.org/fhir/myIG/, then MyProfile documentation will be at: http://example.org/fhir/myIG/StructureDefinition-MyProfile.html. The canonical for MyProfile will be http://example.org/fhir/myIG/StructureDefinition/MyProfile. So here is what I do:

1. Create a StructureDefinition folder at the root of where my IG is served. Now http://example.org/fhir/myIG/StructureDefinition would go there.
2. Create a MyProfile folder under StructureDefinition. Now http://example.org/fhir/myIG/StructureDefinition/MyProfile would go there -- but there's nothing there to serve yet.
3. Create a simple index.html file in the MyProfile folder. Now http://example.org/fhir/myIG/StructureDefinition/MyProfile will serve that index.html file.
4. Put a META directive in that index.html to redirect to MyProfile's documentation page. E.g.,:

<!DOCTYPE html>
<html>
<META http-equiv="refresh" content="0;URL=http://example.org/fhir/myIG/StructureDefinition-MyProfile.html">
<body>
Redirecting to <a href="http://example.org/fhir/myIG/StructureDefinition-MyProfile.html">http://example.org/fhir/myIG/StructureDefinition-MyProfile.html</a>...
</body>
</html>

So... I don't know if that's helpful. But it was kind of fun figuring out. And a script could be written to basically setup all that structure so the redirects work on any standard HTTP server.

Jose Costa Teixeira (Jun 04 2020 at 22:38):

What are PHP or ASP for?

Grahame Grieve (Jun 04 2020 at 22:46):

well, the scripts look at the request on the canonical URL, and decide whether the client wants HTML, JSON, or XML based on the Accept header, and redirects to the appropriate file

Grahame Grieve (Jun 04 2020 at 22:49):

@Chris Moesel that's new but typical of Amazon. I have no idea how I would set that in bulk because the only way I can see how to make it work says

Amazon S3 has a limitation of 50 routing rules per website configuration

50? I routinely have 1000s. And I still don't see how to do different redirects based on the Accept header. So I'll still call this unsupported for now

David Johnson (Jun 04 2020 at 22:53):

It is easy to mistake generated IG content as plain html that could be served by S3. The solution may be a requirements document that indicates the need for redirects by scripting language, required mime types and any other background processes.

Bob Milius (Jun 04 2020 at 23:01):

We used S3 just as a proof-of-concept to share with others to evaluate the IG during development. When we finally deploy it'll be in a dockerized Apache server. Any advice on using the publish.ini file and steps to run the scripts to generate the proper PHP would be nice so we don't have to reinvent it.

Grahame Grieve (Jun 04 2020 at 23:04):

dockerised Apache will work fine.

Grahame Grieve (Jun 04 2020 at 23:04):

you certainly don't want to re-invent it

Grahame Grieve (Jun 04 2020 at 23:04):

I wrote it up for @Jose Costa Teixeira - do you know where that was?

Jose Costa Teixeira (Jun 04 2020 at 23:06):

I think you mean the instructions in the thread that Bob mentions on the top of this thread?

Jose Costa Teixeira (Jun 04 2020 at 23:09):

https://chat.fhir.org/#narrow/stream/179252-IG-creation/topic/IG.20releases/near/196823326

Grahame Grieve (Jun 04 2020 at 23:59):

yes that one thanks

Chris Moesel (Jun 05 2020 at 13:02):

@Grahame Grieve -- I missed that bit about the S3 limit of 50 rules per website. That's just silly. Sorry for the red herring. And yeah, doing different redirects for accepted content type makes it even trickier. My hack solution doesn't cover that either.

Gino Canessa (Jun 05 2020 at 15:28):

Jumping back to CI/CD, I'm working on GitHub action definitions to run SUSHI and IG-Publisher. With those pieces, you can do quite a bit directly in GitHub (e.g., build and publish to GH-Pages on check-in, etc.).

Jose Costa Teixeira (Jun 05 2020 at 15:36):

I'd like to look at that.

Chris Moesel (Jun 05 2020 at 16:02):

I'm working on GitHub action definitions to run SUSHI and IG-Publisher

@Julia Afeltra also did some work on something like this. She didn't have a chance to take it as far as she wanted, but she might have something to share. Julia, is there anything worth sharing from that effort?

Gino Canessa (Jun 05 2020 at 17:03):

Hoping to get back to it later today - once a pass is working I'll put a link to the repo here. That said, once it's working I hope to move it under the HL7 GH org so that it's not locked on me.

Julia Afeltra (Jun 05 2020 at 19:04):

I don't think I have much to share based off of where I got to in the work. I was trying to do similar things (run SUSHI and the IG publisher and then publish to GitHub Pages), and I was able to get an IG onto GitHub pages. Hopefully things have been going smoothly for you as well.

Grahame Grieve (Jun 05 2020 at 20:46):

I understand the desire for this, but it would be great, from my pov, if the builds doing that would continue to update https://fhir.github.io/auto-ig-builder/builds.html and the #committers/notification thread (unless not a public github repo)

Grahame Grieve (Jun 05 2020 at 20:46):

also, you shouldn't need a github action to run sushi if you can run the ig publisher

Gino Canessa (Jun 05 2020 at 20:51):

Grahame Grieve said:

also, you shouldn't need a github action to run sushi if you can run the ig publisher

Yep, figured that out earlier today :-)

Gino Canessa (Jun 05 2020 at 20:59):

For updating the builds.html file.. I think it's possible, but I need to tinker a bit.
First thought is something like a different webhook for auto-ig-builder that collects the information but doesn't trigger a build. Will add to my list for it.

Preston Lee (Jun 09 2020 at 22:30):

I've done the same for S3/CloudFront redirects on projects, though I would think an apache vhost with mod_rewrite rules (or w/nginx) would be cleaner so nothing needs to be generated

Grahame Grieve (Jun 09 2020 at 22:34):

I still generate the rules for servers that support the rules with enough precision, since there's a lot of them. But right now, I'm still doing redirects using PHP for Apache, because last time I couldn't see how to do rewrite rules based on Accept header for Apache

Last updated: Apr 12 2022 at 19:14 UTC