Internal MIT Kerberos functions used by Samba

[Andreas Schneider]

Hello,

as you might know I'm working on migrating Samba AD to use MIT Kerberos
instead of Heimdal Kerberos since several years now.

The MIT Kerberos library has several symbols which are public but do not offer
a prototype in a header file. Samba uses several of those functions to either
implement important parts or to be able to write tests. This document collects
them and will describe its use cases.

We would like to see a public header file to use those functions. We do not
need stable API guarantees.

A warning before the changes happen to those functions would be great!



decode_krb5_setpw_req

Samba implements its own kpasswd server to be able to apply ACLs on the
principal requesting to change a password. To achieve to do a set_password it
needs do decode a set_password request packet.

See source4/kdc/kpasswd-service-mit.c

It would be great to have a stable API for this function.






The following functions are used by source4/torture/krb5/kdc-mit.c. This is a
set of tests that make sure that the KDC responds correctly to certain errors.

decode_krb5_error

If we send incorrect packets, we expect that certain error codes are returned.
We use that functions to decode error packets and make sure the correct error
code is returned by the KDC.

decode_krb5_as_req

This is needed to verfiy that certain aspects which are expected by AD are
sent in the request.

decode_krb5_as_rep

This is used to verify that the KVNO is for a RODC or a normal DC. Or we use
it to check if we request certain enctypes they are also part of the ticket
returned.

decode_krb5_padata_sequence

We have a test which authenticates with a wrong password. In this case we get
an error packet and check the error e-data has KRB5_PADATA_ENC_TIMESTAMP set.

krb5_free_kdc_req

Used to free memory allocated by the decode functions

krb5_free_kdc_rep

Used to free memory allocated by the decode functions

krb5_free_pa_data

Used to free memory allocated by the decode functions

[Greg Hudson]

Thanks again for writing this up.

Post by Andreas Schneider
The MIT Kerberos library has several symbols which are public but do not offer
a prototype in a header file.

Just to be clear, we don't consider functions exported by libkrb5 to be
part of the public API unless they have a prototype. In most cases
those symbols are exported for ease of testing or by use in other parts
of the tree.

[for the Samba kpasswd server]

Post by Andreas Schneider
decode_krb5_setpw_req

The krb5_setpw_req structure is also not public.

If the only reason Samba needs its own kpasswd server is access control,
I would hope that is a temporary need, although we've been very slow to
create a pluggable access control interface for kadmind's password server.

[for the Samba test suite]

Post by Andreas Schneider
decode_krb5_error
decode_krb5_as_req
decode_krb5_as_rep
decode_krb5_padata_sequence
krb5_free_kdc_req
krb5_free_kdc_rep
krb5_free_pa_data

We have krb5_rd_error() which wraps decode_krb5_error(). This and
krb5_mk_error() exist for applications which use KRB_ERROR messages in
their protocols, such as kprop.

One option would be to add similar krb5_rd_kdc_req(), krb5_rd_kdc_rep(),
and krb5_rd_padata() wrappers, or some variation along those lines. The
argument against doing this is that libkrb5 isn't designed to be a
nuts-and-bolts encoding library for test suites. I've done some
preliminary work on a Python library for this purpose, but I stalled out
after implementing the crypto layer.

A more conservative option, in the sense of not making a long-term API
commitment, would be to create a new public header marked as unstable
across releases, containing prototypes for a set of encoding and
decoding functions, and perhaps some formerly private structures such as
krb5_setpw_req.

I would welcome opinions from other Kerberos developers.
_______________________________________________
krbdev mailing list ***@mit.edu
https://mailman.mit.edu/mailman/listinfo/krbdev

[Andreas Schneider]

Post by Greg Hudson
Thanks again for writing this up.

Post by Andreas Schneider
The MIT Kerberos library has several symbols which are public but do not
offer a prototype in a header file.

Just to be clear, we don't consider functions exported by libkrb5 to be
part of the public API unless they have a prototype. In most cases
those symbols are exported for ease of testing or by use in other parts
of the tree.
[for the Samba kpasswd server]

Post by Andreas Schneider
decode_krb5_setpw_req

The krb5_setpw_req structure is also not public.

krb5_error_code decode_krb5_setpw_req(const krb5_data *code,
krb5_data **password_out,
krb5_principal *target_out);

This is the function prototype. It doesn't use any internal structures. So I
do not see how someone would need to use the structure you're talking about.

Post by Greg Hudson
If the only reason Samba needs its own kpasswd server is access control,
I would hope that is a temporary need, although we've been very slow to
create a pluggable access control interface for kadmind's password server.

I would say yes, but I haven't looked into RODC (Read Only Domain Controller)
details yet.

Post by Greg Hudson
[for the Samba test suite]

Post by Andreas Schneider
decode_krb5_error
decode_krb5_as_req
decode_krb5_as_rep
decode_krb5_padata_sequence
krb5_free_kdc_req
krb5_free_kdc_rep
krb5_free_pa_data

We have krb5_rd_error() which wraps decode_krb5_error(). This and
krb5_mk_error() exist for applications which use KRB_ERROR messages in
their protocols, such as kprop.
One option would be to add similar krb5_rd_kdc_req(), krb5_rd_kdc_rep(),
and krb5_rd_padata() wrappers, or some variation along those lines. The
argument against doing this is that libkrb5 isn't designed to be a
nuts-and-bolts encoding library for test suites. I've done some
preliminary work on a Python library for this purpose, but I stalled out
after implementing the crypto layer.

I would be happy to be able to do all of this in python :) I do not think we
need new functions here.

Post by Greg Hudson
A more conservative option, in the sense of not making a long-term API
commitment, would be to create a new public header marked as unstable
across releases, containing prototypes for a set of encoding and
decoding functions, and perhaps some formerly private structures such as
krb5_setpw_req.

Just having a header which states that it will not have a stable API would be
fine with me.

Post by Greg Hudson
I would welcome opinions from other Kerberos developers.

Thanks for considering,



Andreas

[Greg Hudson]

(Continuing a conversation from March 20. Please read if you are
responsible for packaging MIT krb5 for an OS distribution.)

[I wrote:]

Post by Andreas Schneider

Post by Greg Hudson
The krb5_setpw_req structure is also not public.

krb5_error_code decode_krb5_setpw_req(const krb5_data *code,
krb5_data **password_out,
krb5_principal *target_out);
This is the function prototype. It doesn't use any internal structures. So I
do not see how someone would need to use the structure you're talking about.

You're right. The flip side is that we have vague plans to renormalize
decode_krb5_setpw_req() to match other encoders and use the structure
type; see asn1_k_encode.c:1287. That's not really important if we don't
have a stability guarantee for this function, though.

The real reason I followed up was to ask a question about the libkrb5
soname.

For libkadm5clnt, libkadm5srv, and libkdb5, we install a header file
with a comment noting that the API isn't stable, but we do change the
library soname (via LIBMAJOR in Makefile.in) each time we make
incompatible changes to the ABI.

For libkrb5, we haven't changed the soname in a long time; it has sat at
libkrb5.so.3 since release 1.2. My hazy understanding is that changing
the soname would impose a moderate maintenance cost on downstream
distributions, but I don't know the magnitude of that cost.

So if we do install a header file prototyping some semi-public libkrb5
functions, and later change them, do we plan to bump the library soname?
If we do, it imposes some cost; if we don't, I think it basically means
the Samba package breaks if you upgrade libkrb5, which a downstream
distribution might not want to tolerate.
_______________________________________________
krbdev mailing list ***@mit.edu
https://mailman.mit.edu/mailman/listinfo/krbdev

[Simo Sorce]

(Continuing a conversation from March 20.  Please read if you are
responsible for packaging MIT krb5 for an OS distribution.)
[I wrote:]

Post by Andreas Schneider

Post by Greg Hudson
The krb5_setpw_req structure is also not public.

krb5_error_code decode_krb5_setpw_req(const krb5_data *code,
                                      krb5_data **password_out,
                                      krb5_principal *target_out);
This is the function prototype. It doesn't use any internal
structures. So I 
do not see how someone would need to use the structure you're talking about.

You're right.  The flip side is that we have vague plans to
renormalize
decode_krb5_setpw_req() to match other encoders and use the structure
type; see asn1_k_encode.c:1287.  That's not really important if we
don't
have a stability guarantee for this function, though.
The real reason I followed up was to ask a question about the libkrb5
soname.
For libkadm5clnt, libkadm5srv, and libkdb5, we install a header file
with a comment noting that the API isn't stable, but we do change the
library soname (via LIBMAJOR in Makefile.in) each time we make
incompatible changes to the ABI.
For libkrb5, we haven't changed the soname in a long time; it has sat at
libkrb5.so.3 since release 1.2.  My hazy understanding is that
changing
the soname would impose a moderate maintenance cost on downstream
distributions, but I don't know the magnitude of that cost.

For a foundational library like libkrb5 which is linked in a lot of
software I would define the cost high, it basically requires a rebuild
of all dependent code, and it mans incompatibility with existing
binaries, which may or not be rebuildable (source code not always
available or FTBFS issues may arise).

So if we do install a header file prototyping some semi-public
libkrb5 functions, and later change them, do we plan to bump the
library soname?

Well, in theory if you change exposed symbols you should, independently
of whether there are header files or not.
That said, if this is a private API, marked as so, I do not see why you
would bump the soname. I would though, change the symbol name, and
either maintain a compatibility symbol or have the header use some
dload() trick to check if the symbol is available and gracefully fail
if not.

 If we do, it imposes some cost; if we don't, I think it basically
means
the Samba package breaks if you upgrade libkrb5, which a downstream
distribution might not want to tolerate.

This can be handled in other ways:
- symbol versioning
- dload() tricks
- let distributions care for it
- provide a stable wrapper you are ok exposing in the public API/ABI

Simo.
_______________________________________________
krbdev mailing list ***@mit.edu
https://mailman.mit

[Robbie Harwood]

Post by Simo Sorce

Post by Greg Hudson
You're right. The flip side is that we have vague plans to
renormalize decode_krb5_setpw_req() to match other encoders and use
the structure type; see asn1_k_encode.c:1287. That's not really
important if we don't have a stability guarantee for this function,
though.
The real reason I followed up was to ask a question about the libkrb5
soname.
For libkadm5clnt, libkadm5srv, and libkdb5, we install a header file
with a comment noting that the API isn't stable, but we do change the
library soname (via LIBMAJOR in Makefile.in) each time we make
incompatible changes to the ABI.

These require nontrivial work from downstream. The clearest example is
freeipa, which needs to lock to a specific libkdb5 version, and has to
rebuild every time the krb5 version changes.

There's also been trouble in the past with applications doing similar
things to libkadm5*, but we generally consider that their bug for using
them at all, not a krb5 problem, since there's no stability guarantee as
you say.

Post by Simo Sorce

Post by Greg Hudson
For libkrb5, we haven't changed the soname in a long time; it has sat
at libkrb5.so.3 since release 1.2. My hazy understanding is that
changing the soname would impose a moderate maintenance cost on
downstream distributions, but I don't know the magnitude of that
cost.

For a foundational library like libkrb5 which is linked in a lot of
software I would define the cost high, it basically requires a rebuild
of all dependent code, and it mans incompatibility with existing
binaries, which may or not be rebuildable (source code not always
available or FTBFS issues may arise).

Simo is correct here; the cost is very high.

Post by Simo Sorce

Post by Greg Hudson
So if we do install a header file prototyping some semi-public
libkrb5 functions, and later change them, do we plan to bump the
library soname?

Well, in theory if you change exposed symbols you should,
independently of whether there are header files or not. That said, if
this is a private API, marked as so, I do not see why you would bump
the soname. I would though, change the symbol name, and either
maintain a compatibility symbol or have the header use some dload()
trick to check if the symbol is available and gracefully fail if not.

I was thinking a very caveat emptor approach, and just let the
application figure out what to do. Changing the symbol name minimally
(e.g., just put a change number at the end) would work for this.

Post by Simo Sorce

Post by Greg Hudson
If we do, it imposes some cost; if we don't, I think it basically
means the Samba package breaks if you upgrade libkrb5, which a
downstream distribution might not want to tolerate.

- symbol versioning
- dload() tricks
- let distributions care for it
- provide a stable wrapper you are ok exposing in the public API/ABI

I think the intent is for the burden to fall on Samba here, not on krb5,
to ensure that things are working as expected. Samba uses these
functions for test suite purposes, not at runtime, so we're not actually
looking at breaking anything on upgrade.

Andreas, what do you think?

Thanks,
--Robbie

[Andreas Schneider]

Post by Robbie Harwood

Post by Greg Hudson
You're right. The flip side is that we have vague plans to
renormalize decode_krb5_setpw_req() to match other encoders and use
the structure type; see asn1_k_encode.c:1287. That's not really
important if we don't have a stability guarantee for this function,
though.
The real reason I followed up was to ask a question about the libkrb5
soname.
For libkadm5clnt, libkadm5srv, and libkdb5, we install a header file
with a comment noting that the API isn't stable, but we do change the
library soname (via LIBMAJOR in Makefile.in) each time we make
incompatible changes to the ABI.

These require nontrivial work from downstream. The clearest example is
freeipa, which needs to lock to a specific libkdb5 version, and has to
rebuild every time the krb5 version changes.

We will have more incompatilbe changes with KDB in future. I will try to get
as many changes done in the next version but I'm the Samba one man show at RH
so I can not fully focus on that.

Post by Robbie Harwood
I think the intent is for the burden to fall on Samba here, not on krb5,
to ensure that things are working as expected. Samba uses these
functions for test suite purposes, not at runtime, so we're not actually
looking at breaking anything on upgrade.
Andreas, what do you think?

Well mostly they are used for testing, the only thing is the kpasswd function
but I'm fine to add a configure check if that changes. I do not have a problem
here, just a header file with a prototyte is easier to check then defing it on
our own. I do not need API guarantees for these funcitons.


Andreas
_______________________________________________
krbdev mailing list ***@mit.edu
https://mailman.mit.edu/mailman/listinfo/krbdev

[Will Fiveash]

Post by Simo Sorce

Post by Greg Hudson
The real reason I followed up was to ask a question about the libkrb5
soname.
For libkadm5clnt, libkadm5srv, and libkdb5, we install a header file
with a comment noting that the API isn't stable, but we do change the
library soname (via LIBMAJOR in Makefile.in) each time we make
incompatible changes to the ABI.
For libkrb5, we haven't changed the soname in a long time; it has
sat at libkrb5.so.3 since release 1.2.  My hazy understanding is
that changing the soname would impose a moderate maintenance cost on
downstream distributions, but I don't know the magnitude of that
cost.

For a foundational library like libkrb5 which is linked in a lot of
software I would define the cost high, it basically requires a rebuild
of all dependent code, and it mans incompatibility with existing
binaries, which may or not be rebuildable (source code not always
available or FTBFS issues may arise).

I agree with Simo, please do not change the soname of libkrb5.

Post by Simo Sorce

Post by Greg Hudson
So if we do install a header file prototyping some semi-public
libkrb5 functions, and later change them, do we plan to bump the
library soname?

Well, in theory if you change exposed symbols you should, independently
of whether there are header files or not.
That said, if this is a private API, marked as so, I do not see why you
would bump the soname. I would though, change the symbol name, and
either maintain a compatibility symbol or have the header use some
dload() trick to check if the symbol is available and gracefully fail
if not.

Also agree with Simo's suggestions. Backwards runtime compatibility is
a big deal for Solaris. For private APIs, if they change then we can
rebuild the internally delivered binaries that have a dependency on
those interfaces but because we do not want to force customers/third
parties to recompile binaries depending on public APIs Simo's advice
above is what I also recommend.

Post by Simo Sorce

Post by Greg Hudson
 If we do, it imposes some cost; if we don't, I think it basically
means the Samba package breaks if you upgrade libkrb5, which a
downstream distribution might not want to tolerate.

- symbol versioning
- dload() tricks
- let distributions care for it
- provide a stable wrapper you are ok exposing in the public API/ABI

[Benjamin Kaduk]

Post by Simo Sorce

(Continuing a conversation from March 20.  Please read if you are
responsible for packaging MIT krb5 for an OS distribution.)
[...]
The real reason I followed up was to ask a question about the libkrb5
soname.
For libkadm5clnt, libkadm5srv, and libkdb5, we install a header file
with a comment noting that the API isn't stable, but we do change the
library soname (via LIBMAJOR in Makefile.in) each time we make
incompatible changes to the ABI.
For libkrb5, we haven't changed the soname in a long time; it has sat at
libkrb5.so.3 since release 1.2.  My hazy understanding is that
changing
the soname would impose a moderate maintenance cost on downstream
distributions, but I don't know the magnitude of that cost.

For a foundational library like libkrb5 which is linked in a lot of
software I would define the cost high, it basically requires a rebuild
of all dependent code, and it mans incompatibility with existing
binaries, which may or not be rebuildable (source code not always
available or FTBFS issues may arise).

Agreed on the high impact; it would be a pretty extensive library
transition for Debian. (Which, to be clear, there are procedures
for and it is doable in the needed timescale between releases, but
does require effort and coordination.)

Post by Simo Sorce

So if we do install a header file prototyping some semi-public
libkrb5 functions, and later change them, do we plan to bump the
library soname?

Well, in theory if you change exposed symbols you should, independently
of whether there are header files or not.
That said, if this is a private API, marked as so, I do not see why you
would bump the soname. I would though, change the symbol name, and
either maintain a compatibility symbol or have the header use some
dload() trick to check if the symbol is available and gracefully fail
if not.

Having symbols not visible in a public header is useful for us to
convince ftpmaster/release-team of the correctness of our packaging
changes (e.g., when symbols are removed from libkrb5support); I
don't know that we 100% rely on that, but would be cautious about
removing that property.

-Ben
_______________________________________________
krbdev mailing list ***@mit.edu
https://mailman.mit.edu/mailman/listinfo/krbdev