FHIR Chat · SDC : How to show the correct keyboard i.e. alphanumeric ... · questionnaire

Stream: questionnaire

Topic: SDC : How to show the correct keyboard i.e. alphanumeric ...

Kashyap Jois (Sep 02 2021 at 11:23):

We want to implement a phone number widget in the android SDC library. I was wondering whether we should go ahead with support for http://hl7.org/fhir/R4/extension-regex.html or http://hl7.org/fhir/R4/extension-entryformat.html to implement this

The idea is that based on the regex or entry format, the widget will have either a pop up alphanumeric keyboard, or a phone number key pad

eg for a phone number https://developer.android.com/reference/android/text/InputType#TYPE_CLASS_PHONE

Also is there something else that we should be using ?

cc: @Jing Tang

Jing Tang (Sep 02 2021 at 13:42):

so entry format is the human readable version, it basically is just a hint such as "XXX-XXX-XXXX" to occupy the box before the user inputs the phone number. so the library can't really use that to figure out what keyboard to bring up. the issue with regex is that, it's only used for validation, so the user can still type whatever they want... also there's the problem of "parsing" the regex to know that it's a phone number (i.e. [0-9]* and [0123456789]* are equivalent) so it's not a given.

Jing Tang (Sep 02 2021 at 13:45):

wonder if it makes sense for us to start defining "mobile" extensions to inform this type of choices...

Lloyd McKenzie (Sep 02 2021 at 17:20):

The challenge with a specific widget to define phone numbers is that there are so many widely varying expectations around what's acceptable once you take different jurisdictions, extension approaches, etc. into account. A generic widget would allow values that particular forms probably wouldn't want.

Tilo Christ (Sep 02 2021 at 17:48):

I'm wondering whether an extension for Url fields to constrain the protocol would be good? On the one hand I have yearned for an option to constrain it to just "https:", on the other hand, if it is "tel:" it could activate a phone number input and if it is "mailto:" an email input. I was comparing to the "input type" attribute in HTML, and FHIR Questionnaire is missing exactly these two in comparison. I would then leave it up to the implementer to find a sufficiently capable phone number field for their needs. On a browser one would probably map it directly to "input type="tel"", on mobile there are lots of 3rd party libraries for phone number field.

Brian Postlethwaite (Sep 03 2021 at 01:41):

HTML has some hints that do this style of support, and it's something I've been thinking about for a while, but hasn't got to the top of the list to look into, @Paul Lynch ?
Same for email addresses

Kashyap Jois (Sep 06 2021 at 09:21):

Lloyd McKenzie said:

The challenge with a specific widget to define phone numbers is that there are so many widely varying expectations around what's acceptable once you take different jurisdictions, extension approaches, etc. into account. A generic widget would allow values that particular forms probably wouldn't want.

What about extending the item-control and adding a new extension for "number" . That way we can handle even credit card numbers or Personal identification numbers

Brian Postlethwaite (Sep 06 2021 at 09:33):

The ask here is for a way to hint what type of field it is so that the Web controls can show the correct keyboard. And likely other possible behavious etc.

Lloyd McKenzie (Sep 06 2021 at 15:54):

Feel free to make a proposal, but I'm not clear what we can do that will work generically on an international basis

Jose Costa Teixeira (Sep 06 2021 at 16:23):

Brian Postlethwaite said:

The ask here is for a way to hint what type of field it is so that the Web controls can show the correct keyboard. And likely other possible behavious etc.

you mean something that in one's country would have a specific meaning? i.e. when you put "phone" in one country would mean xxx xxx xx xx and in another xxx xxx xxx,...?

Lloyd McKenzie (Sep 06 2021 at 17:36):

Even in a single country - is country code required? Region code? Extensions? What a given UI accepts is likely to vary. Regex and pattern give you a generic way of defining what you want. In theory, given a regex, you could have a control that enforces that. That would be more useful and more generic...

Jose Costa Teixeira (Sep 06 2021 at 19:34):

Indeed. I don't think a valueset exists for that or it would even be sensible to aim at that. Phone number digits are grouped in whatever way people want even within a country. Or within a the same device, if the device handles 2 countries at the same time.

Jose Costa Teixeira (Sep 06 2021 at 19:40):

Going back to the original question, about bringing up the right keyboard for mobile devices, I imagine we could have a "preferredEntryMode" extension, which could be populated with a few values, like "alphaKB" or "numKB" (or "drawingPad", or Neuralink)

If that is sensible, would that be implementation-specific, or something for standard or SDC extension?

Kashyap Jois (Sep 08 2021 at 07:32):

@Lloyd McKenzie any insight on @Jose Costa Teixeira question? Considering the fact that the SDC component of the android FHIR SDK is sticking to the SDC-IG, should this extension be proposed as an addition to that?
Also as a questionnaire designer should I be bothered about which keyboard pops up vs what the question expects as an answer? What kind of extension makes more sense? Something like "preferedEntryMode" eg: "alphaKB", "drawingPad" or "stringType" eg: "number", "email", "signature" etc.

cc: @Jing Tang

Lloyd McKenzie (Sep 08 2021 at 13:03):

Do we have a value set that would work consistently across devices and technologies?

Kashyap Jois (Sep 09 2021 at 07:37):

Nope, not to my knowledge. I will bring this up in the WHO Call we have every Tuesday with implementers of the Android FHIR SDK and WHO Smart Guidelines. I will post the feedback in this chat. Thanks for the quick response @Lloyd McKenzie

Tilo Christ (Sep 09 2021 at 17:58):

Lloyd McKenzie said:

Even in a single country - is country code required? Region code? Extensions? What a given UI accepts is likely to vary. Regex and pattern give you a generic way of defining what you want. In theory, given a regex, you could have a control that enforces that. That would be more useful and more generic...

I think regex is putting the onus of defining a proper pattern on the author of the questionnaire (a clinician?). Not a trivial exercise, that scores of skilled regex wranglers have gotten wrong. I think it makes sense to have something that indicates that this field is meant to be a phone number and then leave it to the implementer of the questionnaire filler to choose a powerful phone number field/validation library. Ultimately, that would give you both ways. If the author of a questionnaire believes they can define the better regex they can still do so through entryFormat, and if they want to hand it to the maker of the filler they can also do it. The filler variant also allows for effects such as a country flag dropdown and normalisation to a canonical number (e.g. "tel:+18884444888x8888").

Tilo Christ (Sep 09 2021 at 18:12):

Kashyap Jois said:

Lloyd McKenzie any insight on Jose Costa Teixeira question? Considering the fact that the SDC component of the android FHIR SDK is sticking to the SDC-IG, should this extension be proposed as an addition to that?
Also as a questionnaire designer should I be bothered about which keyboard pops up vs what the question expects as an answer? What kind of extension makes more sense? Something like "preferedEntryMode" eg: "alphaKB", "drawingPad" or "stringType" eg: "number", "email", "signature" etc.

cc: Jing Tang

I think for "number", "signature", etc. one would not need a preferred entry mode, as this is already expressed by the item types and the keyboard / signature pad can be brought up accordingly. I think the notable exception if you compare to the capabilities of HTML input field hints is the "email" and "tel(ephone number)". HTML can express both as URLs with a special protocol "mailto" and "tel". I am wondering whether something linke the following would make sense:
{ "status": "draft", "resourceType": "Questionnaire", "meta": { "profile": [ "http://hl7.org/fhir/uv/sdc/StructureDefinition/sdc-questionnaire|2.7" ], "tag": [ { "code": "lformsVersion: 29.2.1" } ] }, "item": [ { "type": "url", "required": true, "repeats": true, "linkId": "dayphone", "text": "daytimePhone", "extension": [ { "url": "http://hl7.org/fhir/url-protocol", "valueString": "tel" } ] } ] }

Brian Postlethwaite (Sep 09 2021 at 20:31):

tempFileForShare_20210910-063033.jpg

Brian Postlethwaite (Sep 09 2021 at 20:32):

I wasn't thinking about the protocol stuff, more the keyboard layout the filler uses. And the 2 examples you give are good as with email you get the @ sign specially added, and phone numbers even though a string tweak the keyboard such as this shown on android

Lloyd McKenzie (Sep 09 2021 at 20:32):

Saying "phone number" isn't much help if you don't know what the rules are. Sure, a data entry tool could pick an arbitrary phone number algorithm, but that might allow in either more than is desired or less. If you're concerned about clinicians not getting the regex right, you're free to leverage the ability to point to 'standard' data elements where the rules have been pre-defined by someone who presumably knows how.

If you're using FHIR to capture a phone number, you shouldn't be using URL. We made a very conscious decision to break from the v3 choice of representing phone numbers as URLs - because the URL rules are more stringent than most real-world systems can manage, and sometimes prevent you from saying what you want. A telephone number is a string. Sometimes the proper value for a phone number really is "1-888-NOCOVID" or something like that.

Tilo Christ (Sep 09 2021 at 21:15):

Lloyd McKenzie said:

Saying "phone number" isn't much help if you don't know what the rules are. Sure, a data entry tool could pick an arbitrary phone number algorithm, but that might allow in either more than is desired or less. If you're concerned about clinicians not getting the regex right, you're free to leverage the ability to point to 'standard' data elements where the rules have been pre-defined by someone who presumably knows how.

The 'standard' rules have been pre-defined in fine libraries, such as Google's libphonenumber (https://github.com/google/libphonenumber). It can determine whether a phone number is valid in a given region and pretty-print it, or canonicalise it. Seeing how this weighs in at 200KB of metadata to describe valid phone numbers (plus a rules engine to execute it) in my mind makes it clear that linking this library to a filler might be more feasible than pointing a telephony layperson questionnaire author at a library of regexes.

If you're using FHIR to capture a phone number, you shouldn't be using URL. We made a very conscious decision to break from the v3 choice of representing phone numbers as URLs - because the URL rules are more stringent than most real-world systems can manage, and sometimes prevent you from saying what you want. A telephone number is a string. Sometimes the proper value for a phone number really is "1-888-NOCOVID" or something like that.

Sorry, I'm not a FHIR historian and wasn't aware of that determination when I outlined my thoughts. I've had some exposure to computerised telephony/call-centers and the drivers for auto-dialling are oftentimes based on tel: URLs, so that's what I chose for my example. So if one goes deliberately too far beyond that it will look nice to humans, but potentially break auto-dial. Similar situation if you try to auto-dial a deliberately very poorly formed phone# on a smartphone.

Brian Postlethwaite (Sep 09 2021 at 21:17):

Its the libs and tools that do come native on a platform or browser that I'm looking to tag and enable, not expecting that we'd do it ourselves. But that we could too if the filler wanted.

Lloyd McKenzie (Sep 10 2021 at 14:31):

We have an extension to support autodial. The base data type in FHIR has zero expectation that captured/shared numbers will work for autodial.

Tilo Christ (Sep 10 2021 at 14:41):

Lloyd McKenzie said:

We have an extension to support autodial. The base data type in FHIR has zero expectation that captured/shared numbers will work for autodial.

Can you point me to that? I wasn't aware of it.

Tilo Christ (Sep 10 2021 at 15:00):

Brian Postlethwaite said:

Its the libs and tools that do come native on a platform or browser that I'm looking to tag and enable, not expecting that we'd do it ourselves. But that we could too if the filler wanted.

So between the desire to tag something as a phone#/email for proper keyboard display, but still using a regex to determine its format, would we be looking at something like this? As per @Lloyd McKenzie I have changed the item type to "string", I have introduced the idea of an "entryHint" extension for keyboard control, and use the existing "entryFormat" extension for the regex. I have grabbed the codes for "entryHint" from here: https://www.hl7.org/fhir/codesystem-contact-point-system.html

{
  "status": "draft",
  "resourceType": "Questionnaire",
  "meta": {
    "profile": [
      "http://hl7.org/fhir/uv/sdc/StructureDefinition/sdc-questionnaire|2.7"
    ]
  },
  "item": [
    {
      "type": "string",
      "required": true,
      "repeats": true,
      "linkId": "dayphone",
      "text": "Daytime phone",
      "extension": [
        {
          "url": "http://hl7.org/fhir/StructureDefinition/entryHint",
          "valueCode": "phone"
        },
        {
            "url": "http://hl7.org/fhir/StructureDefinition/entryFormat",
            "valueString": "^(\\([0-9]{3}\\) |[0-9]{3}-)[0-9]{3}-[0-9]{4}$"
        }
      ]
    },
    {
        "type": "string",
        "required": true,
        "repeats": true,
        "linkId": "workmail",
        "text": "Work email",
        "extension": [
          {
            "url": "http://hl7.org/fhir/StructureDefinition/entryHint",
            "valueCode": "email"
          },
          {
              "url": "http://hl7.org/fhir/StructureDefinition/entryFormat",
              "valueString": "(?:[a-z0-9!#$%&'*+/=?^_`{|}~-]+(?:\\.[a-z0-9!#$%&'*+/=?^_`{|}~-]+)*|\"(?:[\\x01-\\x08\\x0b\\x0c\\x0e-\\x1f\\x21\\x23-\\x5b\\x5d-\\x7f]|\\[\\x01-\\x09\\x0b\\x0c\\x0e-\\x7f])*\")@(?:(?:[a-z0-9](?:[a-z0-9-]*[a-z0-9])?\\.)+[a-z0-9](?:[a-z0-9-]*[a-z0-9])?|\\[(?:(?:(2(5[0-5]|[0-4][0-9])|1[0-9][0-9]|[1-9]?[0-9]))\\.){3}(?:(2(5[0-5]|[0-4][0-9])|1[0-9][0-9]|[1-9]?[0-9])|[a-z0-9-]*[a-z0-9]:(?:[\\x01-\\x08\\x0b\\x0c\\x0e-\\x1f\\x21-\\x5a\\x53-\\x7f]|\\[\\x01-\\x09\\x0b\\x0c\\x0e-\\x7f])+)\\])"
          }
        ]
      }

  ]
}

Lloyd McKenzie (Sep 10 2021 at 18:21):

I'd expect entry-hint to more typically say something like "e.g. someone@somewhere.com" or "111-111-1111"

Tilo Christ (Sep 10 2021 at 18:54):

Lloyd McKenzie said:

I'd expect entry-hint to more typically say something like "e.g. someone@somewhere.com" or "111-111-1111"

Sorry, I had mixed up entryFormat and regex extension. entryHint was something to tell the machine which keyboard to pop up. But any idea from a native speaker which name might better convey this is great. Html calls it input type, so I renamed it for now and fixed the mistake with entryFormat vs regex:

{
    "status": "draft",
    "resourceType": "Questionnaire",
    "meta": {
        "profile": [
            "http://hl7.org/fhir/uv/sdc/StructureDefinition/sdc-questionnaire|2.7"
        ]
    },
    "item": [
        {
            "type": "string",
            "required": true,
            "repeats": true,
            "linkId": "dayphone",
            "text": "Daytime phone",
            "extension": [
                {
                    "url": "http://hl7.org/fhir/StructureDefinition/inputType",
                    "valueCode": "phone"
                },
                {
                    "url": "http://hl7.org/fhir/StructureDefinition/entryFormat",
                    "valueString": "(555) 123-4567"
                },
                {
                    "url": "http://hl7.org/fhir/StructureDefinition/regex",
                    "valueString": "^(\\([0-9]{3}\\) |[0-9]{3}-)[0-9]{3}-[0-9]{4}$"
                }
            ]
        },
        {
            "type": "string",
            "required": true,
            "repeats": true,
            "linkId": "workmail",
            "text": "Work email",
            "extension": [
                {
                    "url": "http://hl7.org/fhir/StructureDefinition/inputType",
                    "valueCode": "email"
                },
                {
                    "url": "http://hl7.org/fhir/StructureDefinition/entryFormat",
                    "valueString": "john.doe@email.org"
                },
                {
                    "url": "http://hl7.org/fhir/StructureDefinition/regex",
                    "valueString": "(?:[a-z0-9!#$%&'*+/=?^_`{|}~-]+(?:\\.[a-z0-9!#$%&'*+/=?^_`{|}~-]+)*|\"(?:[\\x01-\\x08\\x0b\\x0c\\x0e-\\x1f\\x21\\x23-\\x5b\\x5d-\\x7f]|\\[\\x01-\\x09\\x0b\\x0c\\x0e-\\x7f])*\")@(?:(?:[a-z0-9](?:[a-z0-9-]*[a-z0-9])?\\.)+[a-z0-9](?:[a-z0-9-]*[a-z0-9])?|\\[(?:(?:(2(5[0-5]|[0-4][0-9])|1[0-9][0-9]|[1-9]?[0-9]))\\.){3}(?:(2(5[0-5]|[0-4][0-9])|1[0-9][0-9]|[1-9]?[0-9])|[a-z0-9-]*[a-z0-9]:(?:[\\x01-\\x08\\x0b\\x0c\\x0e-\\x1f\\x21-\\x5a\\x53-\\x7f]|\\[\\x01-\\x09\\x0b\\x0c\\x0e-\\x7f])+)\\])"
                }
            ]
        }
    ]
}

Lloyd McKenzie (Sep 10 2021 at 21:16):

If we were going to define a keyboard type/input type, we'd need:

• an expectation that it would work across a range of browsers and device types
• an expectation that it would work consistently

If we can come up with a value set that meets those two requirements, I'd be happy to see a change request for us to define such an extension.

Brian Postlethwaite (Sep 12 2021 at 01:26):

There are still several options here and I'm not suggesting we force them all into the same thing.

• keyboard hint (email/number/telephone)
• ghost text
• refex validation
• input mask
  Each has its own contribution to make.

Brian Postlethwaite (Sep 12 2021 at 01:26):

And ignoring them (except regex validation) should be perfectly safe.

Brian Postlethwaite (Sep 13 2021 at 06:59):

This is an example in Xamarin of the keyboard hint texts.
image.png

Tilo Christ (Sep 13 2021 at 18:40):

Brian Postlethwaite said:

This is an example in Xamarin of the keyboard hint texts.
image.png

This looks somewhat like the intersection between Android and iOS. They both add additional other hints (such as a dedicated "Twitter" keyboard on iOS), but those don't look too relevant for typical FHIR use-cases. Staying with your Xamarin example, I am wondering what would be the ones that should be selectable through FHIR? I think "email" and "phone" are good, because they are hints for string fields and there is no obvious other way to state that a string is really an email. But what about "numeric"? FHIR already has decimal, quantity, and integer item types. Should there still be a way to tag a string as being best entered through a "numeric" keyboard? I cannot think of a use-case. If I remove all the ones from the Xamarin list that can be specified through dedicated item types, then only email and phone would remain? "Chat" would make emojis more accessible, but maybe that's too fancy for FHIR?

Brian Postlethwaite (Sep 13 2021 at 20:09):

Depends on the design of the questionnaire, I've had numeric fields that actually needed to be in the string format before (so they could retain the leading 0s) and chat might be applicable for "how are you feeling" self assessment items.

Kashyap Jois (Sep 14 2021 at 13:00):

There are use cases for using the numeric keypad for things other than phone number , for example : Please enter your 16 digit Health Card number / or Identification number.

Tilo Christ (Sep 14 2021 at 19:47):

So with the Xamarin list as the basis, it would be the following:

• Default / Plain text - no need for an explicit hint, this will be the regular keyboard layout for item type "string"
  available on any platform

Proposed FHIR hint name: plain (default - same as not hinting)

• Email - can be added as a keyboard hint to item type "string", will make at-sign more accessible, will adapt predictions
  iOS type "emailAddress", Android: "Email", HTML: input type="email", Flutter: emailAddress

Proposed FHIR hint name: email

• Numeric - can be added as a keyboard hint to item type "string", will make switch keyboard to digits, will adapt predictions
  Also the default for item types "decimal", "quantity", "integer"
  iOS type "decimalPad", Android: "Number", HTML: input type="number", Flutter: number

Proposed FHIR hint name: number

• Telephone - can be added as a keyboard hint to item type "string", will make switch keyboard to phone keypad (digits with 3 letters), will adapt predictions
  iOS type "phonePad", Android: "Phone", HTML: input type="tel", Flutter: phone

Proposed FHIR hint name: phone

• Text / multiline text - no need for an explicit hint, this will be the regular keyboard layout for item type "text", might reconfigure enter key, might affect capitalisation
  iOS type: not available, Android: not available, HTML: input rows="x", Flutter: multiline

Proposed FHIR hint name: (none)

• Url- no need for explicit hint, this will be the regular keyboard layout for item type "url". Makes forward-slash more accessible, adapts predictions
  iOS type: URL, Android: Uri, HTML: input type="url", Flutter: url

Proposed FHIR hint name: url (is there a reason to tag an item of type "string" with this?)

• Chat - can be added as a keyboard hint to item type "string", will make emojis more accessible
  iOS type: not available (type "twitter" does sth. different), Android: TYPE_TEXT_VARIATION_SHORT_MESSAGE, HTML: not available, Flutter: emulation through 3rd party (emoji_picker_plugin)

Proposed FHIR hint name: chat

Proposed extension name: http://hl7.org/fhir/StructureDefinition/inputType, with valueCode, where the code can be any of the hint names proposed above.

Documentation:

• Xamarin: https://andyp.dev/posts/xamarin-forms-essentials-keyboard-master-guide
• Android: https://developer.android.com/reference/kotlin/androidx/compose/ui/text/input/KeyboardType
• iOS: https://developer.apple.com/documentation/uikit/uikeyboardtype
• HTML: https://www.w3schools.com/tags/att_input_type.asp
• Flutter: https://api.flutter.dev/flutter/services/TextInputType-class.html

Tilo Christ (Sep 14 2021 at 19:50):

Lloyd McKenzie said:

If we were going to define a keyboard type/input type, we'd need:

• an expectation that it would work across a range of browsers and device types
• an expectation that it would work consistently

If we can come up with a value set that meets those two requirements, I'd be happy to see a change request for us to define such an extension.

See my analysis on what hinting is available on which popular platforms. Especially email, phone, and number should be very consistently supported.

Lloyd McKenzie (Sep 14 2021 at 19:54):

This sounds solid enough for a formal proposal. Care to submit a tracker item on Jira @Tilo Christ? (And thanks for the analysis work!)

Lloyd McKenzie (Sep 14 2021 at 19:56):

Why no hint for multi-line text? Are we collapsing it into Default/Plain?

Tilo Christ (Sep 14 2021 at 20:06):

I thought FHIR item type "text" was the equivalent of "multi-line" behaviour? In the meanwhile I have also found the missing piece of documentation how to explicitly trigger this changed behaviour on Android (TYPE_TEXT_VARIATION_LONG_MESSAGE), but was generally trying to avoid duplicating things as a hint that can already be triggered by general FHIR item types (thus also the question I had around number and url. For number there were good reasons to also have it for strings, for url I'm not sure).

Tilo Christ (Sep 14 2021 at 20:43):

Lloyd McKenzie said:

This sounds solid enough for a formal proposal. Care to submit a tracker item on Jira Tilo Christ? (And thanks for the analysis work!)

https://jira.hl7.org/browse/FHIR-33675

Brian Postlethwaite (Sep 15 2021 at 02:30):

Yes, text vs string for multiline is what I do too.

Kashyap Jois (Sep 15 2021 at 07:45):

Looks great! Thanks @Tilo Christ

Tilo Christ (Sep 17 2021 at 17:42):

My first stab at implementing the FHIR-33675 "sdc-questionnaire-keyboard" extension:
FHIR-keyboard.gif

Kashyap Jois (Sep 24 2021 at 07:47):

There is another use case that popped up. How do we force uppercase / lowercase etc? Considering the phone ecosystem ? Is it suggested to use the http://hl7.org/fhir/R4/extension-rendering-style.html ?

Lloyd McKenzie (Sep 24 2021 at 14:51):

Style is things like bold, italic, etc., and it applies to the item.text, not the item.answer. If you want to constrain to capturing only lower-case or upper-case data, right now that's a regular expression.

Last updated: Apr 12 2022 at 19:14 UTC