Stream: dotnet
Topic: memory problems with fhir-net-api
Sean McIlvenna (Mar 22 2017 at 18:15):
I am discovering that I have some excessive memory usage related to the fhir-net-api. Has anyone else experienced this? Are there specific things I should be doing to "finish" using the fhir-net-api (dispose the objects, or send them to the gc)?
Brian Postlethwaite (Mar 22 2017 at 20:42):
Do you have any more details on this?
I've done a lot of testing with it and haven't experienced the issue in my server.
And the unit tests that run over all the examples don't exhibit the issues.
Michel Rutten (Mar 23 2017 at 09:26):
@Sean McIlvenna Forge is using the FHIR .NET API and I haven't noticed any excessive memory usage. Can you provide some more details?
Sean McIlvenna (Mar 23 2017 at 14:48):
Sean McIlvenna (Mar 23 2017 at 14:50):
Both of these represent data being used by fhir-net-api.
This seems to be just upon loading the application, not having done any work serializing or deserializing Fhir models yet
Sean McIlvenna (Mar 23 2017 at 14:52):
Sean McIlvenna (Mar 23 2017 at 14:53):
The first relating to left-over instances of String
The second relating to left-over instances of Dictionary<string, Object>
Michel Rutten (Mar 23 2017 at 14:58):
Interesting analysis, what tool are you using? I assume the initial memory pressure is the result of loading the FHIR models from specification.zip.
Sean McIlvenna (Mar 23 2017 at 14:58):
ANT Memory Profiler
Sean McIlvenna (Mar 23 2017 at 14:59):
ANTS Memory Profiler
Sean McIlvenna (Mar 23 2017 at 14:59):
Have had some memory issues as of recently... Trying to nail them down. May not be related to fhir-net-api at all, but this is what I'm seeing in the profiles, so far
Michel Rutten (Mar 23 2017 at 14:59):
ok, we don't have that tool so I can't repeat your analysis.
Michel Rutten (Mar 23 2017 at 15:00):
Wondering what am I looking at? Are these live objects? Or are they ready to be collected?
Sean McIlvenna (Mar 23 2017 at 15:07):
It appears that they ARE collected
Sean McIlvenna (Mar 23 2017 at 15:08):
but they are causing .NET to reserve a ton of memory
Michel Rutten (Mar 23 2017 at 15:10):
Hmm... I'd be more than interested to work together and try to improve this. Currently our dev team is consumed by STU3 update. But I'd love to get back on this.
Ewout Kramer (Mar 23 2017 at 15:16):
Hi Sean, can you tell me what you were doing? Was this in the STU3 or DSTU2 version? Do you know whether this is recent?
Sean McIlvenna (Mar 23 2017 at 15:17):
ok... well, I realize just now that I DO load all profiles at application start so that I don't have to do it mid-request
Sean McIlvenna (Mar 23 2017 at 15:17):
hmmm, might have to re-think that
Sean McIlvenna (Mar 23 2017 at 15:20):
STU3, btw Ewout
Ewout Kramer (Mar 23 2017 at 15:43):
Interesting - STU3 is in alpha but should not really (yet) differ in behaviour. Obviously, it has not been tested in production as far as I know, so keep us in the loop if you find something suspicious. Yes, reading all SDs from profiles-others for example would create quite some objects...
Michel Rutten (Mar 23 2017 at 15:51):
@Sean McIlvenna , I'm wondering what the black dictionary instances in the right column (2nd picture) represent. Can you expand them and/or provide some more details?
Sean McIlvenna (Mar 23 2017 at 15:52):
I tried expanding them, and it didn't show what they contain, or any other helpful info, for that matter
Sean McIlvenna (Mar 23 2017 at 15:52):
=[
Sean McIlvenna (Mar 23 2017 at 15:53):
I think my problem is entirely related to the fact that I'm pre-loading all STU3 profiles
Sean McIlvenna (Mar 23 2017 at 15:53):
I'm working on altering my logic now to separate out all the profiles into individual xml files, and then load each one as I need them
Sean McIlvenna (Mar 23 2017 at 15:53):
not pre-loading them seems to have saved myself about 200mb
Sean McIlvenna (Mar 23 2017 at 16:29):
I take it fhir-net-api STU3 is still not released to nuget?
Sean McIlvenna (Mar 23 2017 at 16:29):
couldn't find it, at least
Michel Rutten (Mar 23 2017 at 16:35):
https://www.nuget.org/packages/Hl7.Fhir.STU3/0.93.0-alpha1
Ewout Kramer (Mar 23 2017 at 17:29):
Hey Sean, yes, and since all profiles carry a *huge* snapshot version, this will result in zillions of objects....
Ewout Kramer (Mar 23 2017 at 17:29):
There is a cached resolver that just caches what you need, maybe that would work?
Sean McIlvenna (Mar 23 2017 at 21:05):
Is that representative of the develop-stu3 branch?
Sean McIlvenna (Mar 23 2017 at 21:05):
0.93.0-alpha1
Brian Postlethwaite (Mar 24 2017 at 05:55):
Just a note on what you loaded in Sean, you might want to trash teh narratives from those resources as they come in if you don't need them as they are quite large according to the info you have there.
Brian Postlethwaite (Mar 24 2017 at 05:56):
or depending on where you got them from (local disk?) can you remove them from your source so doesn't even get serialized too.
Last updated: Apr 12 2022 at 19:14 UTC
