Roadmap to EST support #859
Comments
|
@sbernard31 I am currently modifying the web user interface for bootstrap server. When I click on "Add new client bootstrap configuration" it displays a configuration. Let's assume we have a device with device certificate which is connecting (before auto registration support is ready) I was thinking I would do following:
Now related question:
|
|
The LWM2M bootstrap server is just a demo and is clearly not perfect. I think it should not really aims to be like a production ready server but maybe more a way to discover LWM2M, Leshan and its API but also to do quick interoperability tests. About some question about the "LWM2M Bootstrap Server" tab : The read only field and private key value are here because this is the
This is the client certificate we will write in security object. My understanding :
I'm not sure you will like this answer 😬 About PEM support, I don't know. Dp you want to support both format in demos or replace DER by PEM ? If we decide to add it, this should be done in a consistent way :
All of this make me think that maybe we are not really ready to put EST now. 🤔 |
|
About Certificate Usage support: I believe I have it implemented in here:
And you actually need some other changes in the branch there to actually too. I am happy to make PR from that branch but I am wondering should it be split a bit first? And you probably want more testing ;) |
You actually support PEM format already -- just may have not noticed it -- code transparently parses both PEM and DER. DER format has a limitation that it does not support one to carry certificate chain where as PEM format has agreement that you can put multiple PEM blocks in sequence to specify certificate chain. Even this is supported out-of-the-box but it just needs to be taken in use which is also supported in: I am happy to support both DER and PEM. But in order to define chain you commonly just use PEM format and usually just copy'n'paste it either to the field or to the file. Another common way to do it in Java application is to utilize Java/pkcs12 key/truststore. I have support for Java trust store in Californium way in that branch. So my question is it OK to support both DER (as hex) and PEM in demo application web user interfaces. Tools will be synchronized for same functionality as I have done already for other improvements in that branch. I also have some code for use Java keystore (utilizing californium) to provide private key and certificate and its chain -- but that is not in the branch as I was not sure this was something that you wanted or not -- but I can bring that support also back. What is good in here that internally the code is using X509Certificate class which abstract way how the certificate was loaded. And you can convert that to DER byte code or to PEM. Only "ugly" thing I had to do is that when one configures certificate chain from command line (or gets it from EST) it stores it in Security object as PEM string (instead of DER byte array). What is good in here that it handles transparently both cases so not a big deal. LWM2M only uses DER to carry it within security object so it is currently limited to only one level CA setup. Only with EST one can carry whole chain. Probably something that should be improved in LWM2M protocol itself. |
Isn't that in the LWM2M Server tab as that also contains the server to write?
No I want to write both configs :) How I understand is that bootstrap config tab defines how one identifies the connecting client. Then the server tab specifies contents for the security object to be written into the client. But as an example if you configure authentication with X.509 certificate -- bootstrap server shouldn't have a need to have client's private key at all and now this field is mandatory in UI. I don't see a way in client demo to have access to store to fetch it either (and I don't think that is probably even a good idea). I believe the private key in bootstrap configuration in X.509 certificate authentication should be removed so that people don't get confused about it and in reality one shouldn't even have a means the get it from device. (In server tab it should be kept when client is configured with X.509 security config with bootstrap writes) I can improve the situation -- I anyway need to modify it to support for EST which I have started in this branch (expect that branch to be windy): |
|
Here some quick answers, but I think we should not focus on this. About PEM, we could add it later (storing PEM in security object sounds really out of the spec and asI first sight I don't really like the idea but maybe this would be another case where we need to not respect the standard to make it works in the real life ... 😞 ) About bootstrap config,
=> this is not the idea of current bs server demo (even if this sounds probably a bad idea to you). Both bootstrap and server config tabs aim to show what will be written in the security and server object for both servers on the device. (bootstrap and LWM2M server). We first clear (delete request) object 0 and 1, then we rewrite it with given credentials. (There is a lot of issue about that but again this is not a real server and the idea was more to be able to test if bootstrap delete and bootstrap write request works as expected). But talking about bootstrap server demo sounds premature to me, again my feeling that we want to address too many subject at the same time. LWM2M v1.0 has a very limited support of x509. As I understand the specification, in all mode except EST, the secret can be written (and so known) by the bootstrap server. (I know this could be troubling) So how could you handle bootstrap to make the device uses x509. For client credentials, you need to write the private key and the client certificate but there is no information about how server should use it and there is a limitation : "no way to set the certchain" (which sounds still there in LWM2M 1.1) ...
In Leshan :
Do you agree with the current situation ? |
|
@sbernard31 About current state: I believe there is now some confusion so I did a test sequence with PSK to make it simple and it seems to match my understanding what I tried to describe up there: So in summary:
Do you agree? I believe this is how logically one would configure it with PSK. |
|
@sbernard31 please don't be alarmed even if I talk in the future as I try to get the concept working with Leshan -- I have a need to proof it and then we can slice it to smaller pieces. However I think if I ask advice how things should be implemented it makes integration process a bit easier in future. I don't intend to do PR with all in one go. I believe I am missing only coap-est -> est proxy at this time for server side. Don't know yet what kind of problems that brings up. |
OK, thanks to let me know that. 🙏
Yes and no 😁 In a better world, the configuration should be :
About defining bootstrap configuration UI we can imagine several step
By the way #437, is something needed by OSCORE and maybe by you too so I should maybe put it in my priority list 🤔 ? |
|
@sbernard31 I have a PoC in test code base for CoAP-EST <-> EST proxy code in Java. Next thing is to start figuring out how to put all that in Leshan code base. One of the crucial things is the HTTP(S) Request Library. As there are quite many available I just used OkHttp to test it out but that might not be something you prefer to have in Leshan. Do you have preferences for the library? |
This is available at https://github.com/dachaac/leshan/commits/est-support , correct ?
You want me to look at code and tell you if I have idea how to integrate this in Leshan ? (I ask because looking at so much code is time-consuming and want to be sure first)
I do not have real preference, but currently we have some constraints :
|
|
About Java versions. Java 7 has been EOL since 2015 (for paid customers said to be up till 07/2022). Java 8 is LTS release and is about to be EOL at end of 2020 (or 2019). With extended support contract perhaps till 2030. Current LTS release is Java 11. Don't know what kind of effort supporting Java 11 would be but based on those EOL's that sounds like good target for next major version. |
|
About HTTP Request Library. Apache has HttpClient: Which seems to be Java 7 compliant. Example code: Don't know how good this library is but can try it if it looks/feels better than OkHttp. |
|
About CoAP-EST<->EST PoC Code. It is not yet there as I wanted to have easy to hack environment for coding so I took californium's DTLS example server and client and started playing with it. It is somewhat ugly hack ;) but if you want to see it -- suppose I could push it on github. Tested the PoC client code -> libest's CoAP-EST backend and it seems to be functionally matching vs. PoC client -> PoC server code -> libest's EST server. Making it production quality code is yet another thing to ponder thou -- as I took a bit shortcut to just test the real beef. I can look around and give first hypothesis what it could be and push somewhat tested solution on that repo/branch. |
|
java7: httpclient: |
|
Just a warning note about Apache HttpClient -- seems a bit unpolished from usability point of view -- doesn't handle error cases as nicely as OkHttp did. As an example libest has a problem -- it closed the server connection in between Basic Auth requests even thou it should be persistent (one could say that it is a bug in libest) and it doesn't know how to recover. Workaround was to do add credentials to auth cache before so that only one request is sent with basic auth right away. I will use this for now but if you have some other options those might actually be better. When looking around I noticed that Java 11 actually has: Don't know how good that is but at least they have got that into the standard library so it should be at least though a bit more. |
8000
|
Maybe it would not be easy to have a modern API + support of java 7 😅. |
|
My development branch can handle (in a bit dirty way) bootstrapping request with X.509 certificate and then when configured for EST mode from Leshan's bootstrap server's Web UI then it does following flow bootstrap->coap-est-server->lwm2m server. It has still problems in code quality and handling error cases and handling persistence of bootstrapped client certificates and ca certificates. And does not support delayed certificate generation as that includes retrying the process. Does not support PoP support in challenge password. CoAP-EST Registrar->EST server connection with support for client certificate authentication has not been tested only basic auth has been tested. It also brings new dependencies from Apache HTTP Client/codecs and BouncyCastle for CSR generation. If someone plays with it -- word of warning with Cisco's libest -- it needed some fixes in order for it to operate in better fashion. I can try to clean it up my changes and push it on github later. More commercial grade EST server should work better. Also the certificate setup is a bit deep and I will publish them a bit later on my test certificate scripts repo. With libest got once a situation that Apache HTTP Client debug was claiming proper sending of message and libest claiming received completely different request (previously sent request) -- I hope that this is a problem of libest and not Apache HTTP Client. Didn't have debugger plugged in for libest at that time so could not verify that. CoAP-EST support is missing server side key generation facilities as that would require multipart sending in CoAP which seems to be missing from californium/leshan. This may also require one to use BouncyCaste in more core Leshan libraries as one may need to translate a bit requests. Sever side key generation is also defined OPTIONAL so we can also skip implementing that if you think it is good enough without that support: Then again -- one could use that within leshan to auto generate certificates when configuring security objects with X.509 certificates on server side or so. CoAP service discovery for CoAP-EST is not supported. I suppose there is no CoAP discovery support in Leshan. |
|
I had a short look at RFC 8710 - multipart content-format. At least in the past, californium handles the payload in an agnostic way, it just offers the accept/content-type option and the payload bytes. So, I'm not sure, if it fits into Californium and when I find the time to add it. |
We succeed to limit dependencies with Leshan 1.0. It sounds this will be difficult to do the same for Leshan 2.0. 😕
If I read rfc corrrectly, multipart content format is just a new content format + a way to encode/decode a payload using CBOR.
As I don't know what it is, I suppose there is not.
I know I repeat myself 😅 but I still think that that we should move forward step by step and so begin with the minimal feature set and then add new feature if needed. |
Not really. I tried to check the old bouncy-castle CQs. As far as I remember, they have been withdrawn. Unfortunately, I currently couldn't login to verify that ;-(. |
|
@dachaac I asked to OMA about your certchain concerns : OpenMobileAlliance/OMA_LwM2M_for_Developers#502 |
|
Some good news (IP-Zilla works again :-) ): There is now a license certified for |
|
@sbernard31 There is a need to store state information from LeshanClientDemo. Now that we have Jackson available we could use YAML or JSON for serializing state to the file. In this state we could store received configuration during bootstrap and as an example EST objects. Other thing that probably would be very useful is actually moving this command line configuration to configuration file. I am starting to think that it might make it even cleaner and more easier to understand now that there is going to be more flexibility also for bootstrap configuration writing. Using YAML seems to be rather common today for things like configuration files. What do you think if we would create a configuration file which syntax would be somewhat common in *Demo? And start reducing the command line arguments. |
|
@sbernard31 Other thing that I was thinking is that there should be persistent device inventory. I am thinking that device configurations between bootstrap server and normal lwm2m server should be shared so that one could maintain them a bit easier and also to support device auto registration when device is connecting with X.509 certificates. There is already a Redis support but I haven't really looked too deep in that code. Is this intended to be used for this? If I am not mistaken Redis is not exactly meant for configuration store but more like a fast runtime cache with atomicity in a cluster. |
If a persistent state is needed, maybe an interface should be define this way the user can persist as wanted.
No strong opinion but I would prefer to not add new dependencies. (so probably not YAML 😕)
There is an idea about change command line library using picocli for demo : #569
My first feeling, so not set in stone : |
|
I have wrote down instructions on how to test EST support with my est-support branch in here: |
|
Device auto registration will need some kind of shared persistent storage between LWM2M Bootsrap server and LWM2M Server so I will do experiment with shared persistent storage (mongodb probably) and as that needs a bit more configuration I will experiment with YAML file. When done I will push this to custom branch and we can see how it looks. About Picocli -- don't really have opinion is it good or bad -- at least it can be shiny. Seems to provide annotation automata if that is something you wish to have. About configuration I currently think that there would need to be more levels than one to make it easily understandable for grouping things and if needed extendable. As an example for trust store there could be different options how to get them: trust-store:
type: path
path: /path/to/caor: trust-store:
type: file
path: /path/to/cacerts.pemor: trust-store:
type: pkcs12
path: /path/trust-store.p12
password: changeitThen there might be custom trust store for EST server connectivity: est:
trust-store:
<same as normal trust-store> |
I was talking about picocli because, it provides free config file from command line option and so it could maybe cover the config needs.
Not sure to get what you call "auto registration". |
|
With auto registration in here I mean that:
And whole this chain would not require any manual intervention from operator to accept the device into the system. This can only work when Manufacturer's CA is pre-registered into the service and possible policies has been configured. I see part of this as out-of-scope for Leshan but in order to figure out Leshan parts it needs to be tested. If you are happy with those other changes then we can of course consider merging them here too -- but let's see how it first looks. |
Yes. I want to experiment with YAML on how that works first -- if picocli works with the same I don't have anything against that. We can even try both and see how they compare. |
|
Reworked my est-support test branch on top of master. |
|
Added IEEE Secure Device Identity (IDevID) support. Updated the instructions and fixed end point name to be valid |
|
@dachaac we didn't get any news from you since a while now (something 3 or 4 month). |
|
(There some works maybe indirectly related to this which was not merged at #896 (comment)) |
|
As this one is maybe a bit long discussion for some one which would like to start on this subject I create a new one (#1135) which point to this one. |
EST support is partly supported already in LWM2M specification 1.0.2 but is further improved in LWM2M specification 1.1.(1).
I have started writing wiki page with a bit more details in here:
https://github.com/dachaac/leshan/wiki/Improving-support-for-LWM2M-with-Certificates
Will try to make as small changes as possible as PRs to make integration work easier and also to get earlier feedback from other developers.
Will try to tag all related PRs with this ticket.
I believe the goal is achieved when the EST functionality is matching the latest released LWM2M specification (1.1.1 at the moment).
The text was updated successfully, but these errors were encountered: