Stream: committers
Topic: txCache is out
Grahame Grieve (Nov 10 2021 at 06:15):
ok. after scheming with Josh, I came up with a system for handling tx-cache files that means they'll no longer be in git / github. Here's how things are going to work:
- there'll still be a tx-cache, and it will live in {user.home}\.fhir\tx-cache
- there'll be separate caches in that folder by {org}\{repo}\{branch}
- if the IG being built isn't linked to github, then the tx-cache will be in {root}\temp\tx-cache, and the expectation is that you don't commit to whatever version control you're using
- when the build runs (both main FHIR build, and IG build), it will look at the location for the tx-cache, and if the cache is out of date (combination of version flag in the code and on tx.fhir.org), it will wipe it.
- when the build runs (both main FHIR build, and IG build), if the IG is linked to GH, and the cache is empty, it will initialise the cache from http://tx.fhir.org/tx-cache/{org}/{repo}/{branch}.zip, and if that doesn't exist, it'll try http://tx.fhir.org/tx-cache/{org}/{repo}/default.zip. If neither are found, it will run with an empty cache
- when the ci-build runs, at the end of a successful run, it will update http://tx.fhir.org/tx-cache/{org}/{repo}/{branch.zip} with the cache it built. If it's for main or master, will also update the default.zip
- Then anyone doing a build will pick that up if they start with an empty cache
- there's a log of what's getting posted to the server at http://tx.fhir.org/tx-cache/log.txt
- there's 450GB on the server for the cache. If it starts getting too big, I'll start pruning old caches, but the biggest cache I've seen so far is 9MB zipped, so I think that's a wee way off in the future
Grahame Grieve (Nov 10 2021 at 06:17):
Note: this system isn't fully operational yet - another day or two, and then we'll beta test it on the main build - once I'm happy with that, I'll roll it out to the IG build too.
Rob Hausam (Dec 22 2021 at 04:15):
@Grahame Grieve It looks like this is not happening yet for IGs? When I rebuild the IG (IPS in this case) with an absent .fhir\tx-cache and input-cache/txcache, it still rebuilds the cache in input-cache/txcache. And I don't see a log entry being added for it.
Grahame Grieve (Dec 22 2021 at 04:21):
I haven't got to that yet
Rob Hausam (Dec 22 2021 at 04:25):
Ok. I assume that will happen at some point. In the meantime, does it make sense with an IG to still commit the txcache files - or not? If we remove the txcache from version control, then everyone needs to maintain their own. But maybe that's not a bad idea?
Jean Duteau (Dec 22 2021 at 04:33):
i never commit the txcache files for my IGs. I admittedly don't have IGs the size of US Core or UTG, but I've built US Core before and it wasn't a hassle not having the txcache.
Rob Hausam (Dec 22 2021 at 04:39):
That makes sense. I think I will start doing the same. I've just set it up and have been in the habit of committing them as that was the recommendation before.
Last updated: Apr 12 2022 at 19:14 UTC
