krb5.conf and 32 vs 64-bit plugins

[Tomas Kuthan]

Hi team,

in Solaris we deliver 32-bit Kerberos libraries under /usr/lib and
64-bit libraries under /usr/lib/64. The same holds for plugins, which
reside in /usr/lib/krb5/plugins and /usr/lib/64/krb5/plugins respectively.

Specifying a plugin in krb5.conf works fine when relative paths and
default plugin_base_dir are used. The plugin_base_dir defaults to
/usr/lib/krb5/plugins on 32-bit and to /usr/lib/64/krb5/plugins on
64-bit and plugins with the correct ISA are used.

Things start falling apart, when user would like to either specify full
path to the plugin or set a non-default plugin_base_dir in their
krb5.conf. In that case only one of the paths can be specified in
krb5.conf, meaning plugins would fail dlopen-ing on the other architecture.

We would like to solve that by supporting $ISA place holder in the path,
that would translate to '/64/' on 64-bit and to '/' on 32-bit. Hence the
following (artificial) example would work fine for both:

module = pkinit:/lib/$ISA/site/preauth/pkinit.so

Would MIT be willing to accept a patch implementing something along
these lines?

If yes, I would prepare a platform-independent fix.
FYI, attached is a quick unix-only patch.

Thanks,
Tomas

[Roland Mainz]

Post by Tomas Kuthan
Hi team,
in Solaris we deliver 32-bit Kerberos libraries under /usr/lib and 64-bit
libraries under /usr/lib/64. The same holds for plugins, which reside in
/usr/lib/krb5/plugins and /usr/lib/64/krb5/plugins respectively.
Specifying a plugin in krb5.conf works fine when relative paths and default
plugin_base_dir are used. The plugin_base_dir defaults to
/usr/lib/krb5/plugins on 32-bit and to /usr/lib/64/krb5/plugins on 64-bit
and plugins with the correct ISA are used.
Things start falling apart, when user would like to either specify full path
to the plugin or set a non-default plugin_base_dir in their krb5.conf. In
that case only one of the paths can be specified in krb5.conf, meaning
plugins would fail dlopen-ing on the other architecture.

Note that KRB5_CONF and KRB5_KDC_PROFILE can be a (POSIX-shell style)
$PATH, so in *theory* you could provide ISA-specifc configs that
way... :-)

Post by Tomas Kuthan
We would like to solve that by supporting $ISA place holder in the path,
that would translate to '/64/' on 64-bit and to '/' on 32-bit. Hence the
module = pkinit:/lib/$ISA/site/preauth/pkinit.so
Would MIT be willing to accept a patch implementing something along these
lines?
If yes, I would prepare a platform-independent fix.
FYI, attached is a quick unix-only patch.

IMHO it would be better to look at the options provided by the POSIX
standard (e.g. see $ ksh -c 'builtin uname ; uname --help #) for a
list of keys.
Otherwise it will be tricky from the scripting (or any POSIX API-based
application) to figure out the correct values (remember, there isn't
only 32bit and 64bit, for example on Solaris/SPARC you have more
architectural choices (see $ isalist # output and the isaexec stuff))
- unless you want to either hardcode the values for every possible
combination OR limit the choices to 32bit vs. 64bit (not wise - see
SPARC example).

----

Bye,
Roland

[Tomas Kuthan]

Post by Roland Mainz

Post by Tomas Kuthan
Hi team,
in Solaris we deliver 32-bit Kerberos libraries under /usr/lib and 64-bit
libraries under /usr/lib/64. The same holds for plugins, which reside in
/usr/lib/krb5/plugins and /usr/lib/64/krb5/plugins respectively.
Specifying a plugin in krb5.conf works fine when relative paths and default
plugin_base_dir are used. The plugin_base_dir defaults to
/usr/lib/krb5/plugins on 32-bit and to /usr/lib/64/krb5/plugins on 64-bit
and plugins with the correct ISA are used.
Things start falling apart, when user would like to either specify full path
to the plugin or set a non-default plugin_base_dir in their krb5.conf. In
that case only one of the paths can be specified in krb5.conf, meaning
plugins would fail dlopen-ing on the other architecture.

Note that KRB5_CONF and KRB5_KDC_PROFILE can be a (POSIX-shell style)
$PATH, so in *theory* you could provide ISA-specifc configs that
way... :-)

Indeed, in theory.

Post by Roland Mainz

Post by Tomas Kuthan
We would like to solve that by supporting $ISA place holder in the path,
that would translate to '/64/' on 64-bit and to '/' on 32-bit. Hence the
module = pkinit:/lib/$ISA/site/preauth/pkinit.so
Would MIT be willing to accept a patch implementing something along these
lines?
If yes, I would prepare a platform-independent fix.
FYI, attached is a quick unix-only patch.

IMHO it would be better to look at the options provided by the POSIX
standard (e.g. see $ ksh -c 'builtin uname ; uname --help #) for a
list of keys.
Otherwise it will be tricky from the scripting (or any POSIX API-based
application) to figure out the correct values (remember, there isn't
only 32bit and 64bit, for example on Solaris/SPARC you have more
architectural choices (see $ isalist # output and the isaexec stuff))
- unless you want to either hardcode the values for every possible
combination OR limit the choices to 32bit vs. 64bit (not wise - see
SPARC example).

I might be misunderstanding you here, but limiting the choices to 32bit
vs. 64bit is exactly what I want.

In the example above, I'd like

module = pkinit:/lib/$ISA/site/preauth/pkinit.so

to translate to /lib/site/preauth/pkinit.so on 32-bit (i386 or sparc)
and to translate to /lib/64/site/preauth/pkinit.so on 64-bit (amd64 or
sparcv9).

This would allow users to specify custom plugins from outside of default
plugin_base_dir in krb5.conf and have it work with both 32-bit and
64-bit applications.

Thanks,
Tomas
_______________________________________________
krbdev mailing list ***@mit.edu
https://mailman.mit.edu/mailman/listinfo/krbdev

[Roland Mainz]

Post by Tomas Kuthan

Post by Roland Mainz

Post by Tomas Kuthan
Hi team,
in Solaris we deliver 32-bit Kerberos libraries under /usr/lib and 64-bit
libraries under /usr/lib/64. The same holds for plugins, which reside in
/usr/lib/krb5/plugins and /usr/lib/64/krb5/plugins respectively.
Specifying a plugin in krb5.conf works fine when relative paths and default
plugin_base_dir are used. The plugin_base_dir defaults to
/usr/lib/krb5/plugins on 32-bit and to /usr/lib/64/krb5/plugins on 64-bit
and plugins with the correct ISA are used.
Things start falling apart, when user would like to either specify full path
to the plugin or set a non-default plugin_base_dir in their krb5.conf. In
that case only one of the paths can be specified in krb5.conf, meaning
plugins would fail dlopen-ing on the other architecture.

Note that KRB5_CONF and KRB5_KDC_PROFILE can be a (POSIX-shell style)
$PATH, so in *theory* you could provide ISA-specifc configs that
way... :-)

Indeed, in theory.

Yes, not nice, but something which can be used without having to go
through PSARC ...

Post by Tomas Kuthan

Post by Roland Mainz

Post by Tomas Kuthan
We would like to solve that by supporting $ISA place holder in the path,
that would translate to '/64/' on 64-bit and to '/' on 32-bit. Hence the
module = pkinit:/lib/$ISA/site/preauth/pkinit.so
Would MIT be willing to accept a patch implementing something along these
lines?
If yes, I would prepare a platform-independent fix.
FYI, attached is a quick unix-only patch.

IMHO it would be better to look at the options provided by the POSIX
standard (e.g. see $ ksh -c 'builtin uname ; uname --help #) for a
list of keys.
Otherwise it will be tricky from the scripting (or any POSIX API-based
application) to figure out the correct values (remember, there isn't
only 32bit and 64bit, for example on Solaris/SPARC you have more
architectural choices (see $ isalist # output and the isaexec stuff))
- unless you want to either hardcode the values for every possible
combination OR limit the choices to 32bit vs. 64bit (not wise - see
SPARC example).

I might be misunderstanding you here, but limiting the choices to 32bit vs.
64bit is exactly what I want.

Erm... this will ONLY work for x86/AMD64 but NOT for SPARC because
SPARC has more than two ISAs/architectures (and technically if you add
SSE/SSE/etc. to the mix then it will be a problem for x86/AMD64, too)
...
... please take a look at the output of the isalist(1m) utility on
Solaris and then you'll see what I mean.

If you take a look at bugster (yes yes, I've been a SUN'y too long
ago) or whatever Oracle forced you to use as bug database these days
there is a common pattern: 1. Engineers add 32/64bit directories 2.
Few months later said engineers are slapped over by another devel
group, engineering management or PSARC because "...only 32bit/64bit
has been accounted for, not the while list of ISAs on SPARC...".
My suggestion is NOT to repeat that pattern and just make room for
more than two ISAs ([1]).

[1]=And "yes", I know that the runtime linker is now able to make ISA
choices (like done for libc, replacing the Solaris 10/OpenSolaris
solution of using lofs to mount the right ISA libc over /lib/libc.so
...) ... but I am not sure whether this is a PSARC "contracted"
interface or a public one...

----

Bye,
Roland

[Tomas Kuthan]

Post by Roland Mainz

Post by Tomas Kuthan

Post by Roland Mainz

Post by Tomas Kuthan
Hi team,
in Solaris we deliver 32-bit Kerberos libraries under /usr/lib and 64-bit
libraries under /usr/lib/64. The same holds for plugins, which reside in
/usr/lib/krb5/plugins and /usr/lib/64/krb5/plugins respectively.
Specifying a plugin in krb5.conf works fine when relative paths and default
plugin_base_dir are used. The plugin_base_dir defaults to
/usr/lib/krb5/plugins on 32-bit and to /usr/lib/64/krb5/plugins on 64-bit
and plugins with the correct ISA are used.
Things start falling apart, when user would like to either specify full path
to the plugin or set a non-default plugin_base_dir in their krb5.conf. In
that case only one of the paths can be specified in krb5.conf, meaning
plugins would fail dlopen-ing on the other architecture.

Note that KRB5_CONF and KRB5_KDC_PROFILE can be a (POSIX-shell style)
$PATH, so in *theory* you could provide ISA-specifc configs that
way... :-)

Indeed, in theory.

Yes, not nice, but something which can be used without having to go
through PSARC ...

Post by Tomas Kuthan

Post by Roland Mainz

Post by Tomas Kuthan
We would like to solve that by supporting $ISA place holder in the path,
that would translate to '/64/' on 64-bit and to '/' on 32-bit. Hence the
module = pkinit:/lib/$ISA/site/preauth/pkinit.so
Would MIT be willing to accept a patch implementing something along these
lines?
If yes, I would prepare a platform-independent fix.
FYI, attached is a quick unix-only patch.

IMHO it would be better to look at the options provided by the POSIX
standard (e.g. see $ ksh -c 'builtin uname ; uname --help #) for a
list of keys.
Otherwise it will be tricky from the scripting (or any POSIX API-based
application) to figure out the correct values (remember, there isn't
only 32bit and 64bit, for example on Solaris/SPARC you have more
architectural choices (see $ isalist # output and the isaexec stuff))
- unless you want to either hardcode the values for every possible
combination OR limit the choices to 32bit vs. 64bit (not wise - see
SPARC example).

I might be misunderstanding you here, but limiting the choices to 32bit vs.
64bit is exactly what I want.

Erm... this will ONLY work for x86/AMD64 but NOT for SPARC because
SPARC has more than two ISAs/architectures (and technically if you add
SSE/SSE/etc. to the mix then it will be a problem for x86/AMD64, too)
...
... please take a look at the output of the isalist(1m) utility on
Solaris and then you'll see what I mean.
If you take a look at bugster (yes yes, I've been a SUN'y too long
ago) or whatever Oracle forced you to use as bug database these days
there is a common pattern: 1. Engineers add 32/64bit directories 2.
Few months later said engineers are slapped over by another devel
group, engineering management or PSARC because "...only 32bit/64bit
has been accounted for, not the while list of ISAs on SPARC...".
My suggestion is NOT to repeat that pattern and just make room for
more than two ISAs ([1]).
[1]=And "yes", I know that the runtime linker is now able to make ISA
choices (like done for libc, replacing the Solaris 10/OpenSolaris
solution of using lofs to mount the right ISA libc over /lib/libc.so
...) ... but I am not sure whether this is a PSARC "contracted"
interface or a public one...
----
Bye,
Roland

Hi Roland,

on Solaris, instruction set specific shared objects have been deprecated
in favor of capabilities, see S11.2 Linkers and Libraries guide:

Note - This token is obsolete, and might be removed in a future
release of Oracle Solaris. See “Capability Specific Shared Objects”
on page 253 for the recommended technique for handling instruction
set extensions.

All the evidence I was able to gather suggests, that on Solaris alone
32/64-bit distinction in paths is sufficient. But for multi-platform it
is a different story I guess.

Thanks,
Tomas
_______________________________________________
krbdev mailing list ***@mit.edu
https://mailman.mit.edu/mailman/listi

[Greg Hudson]

Post by Tomas Kuthan
We would like to solve that by supporting $ISA place holder in the path,
that would translate to '/64/' on 64-bit and to '/' on 32-bit. Hence the
module = pkinit:/lib/$ISA/site/preauth/pkinit.so
Would MIT be willing to accept a patch implementing something along
these lines?

My concern is that different operating systems have done different
things with 32-bit and 64-bit libraries, even within the space of Linux
distributions. Rather than a solution narrowly tailored to one
operating system, I would like a plan which can cover a broad range of
operating systems, preferrably one which doesn't add a lot of complexity
to the code.
_______________________________________________
krbdev mailing list ***@mit.edu
https://mailman.mit.edu/mailman/listinfo/krbdev

[Tomas Kuthan]

Post by Greg Hudson

Post by Tomas Kuthan
We would like to solve that by supporting $ISA place holder in the path,
that would translate to '/64/' on 64-bit and to '/' on 32-bit. Hence the
module = pkinit:/lib/$ISA/site/preauth/pkinit.so
Would MIT be willing to accept a patch implementing something along
these lines?

My concern is that different operating systems have done different
things with 32-bit and 64-bit libraries, even within the space of Linux
distributions. Rather than a solution narrowly tailored to one
operating system, I would like a plan which can cover a broad range of
operating systems, preferrably one which doesn't add a lot of complexity
to the code.

Hi Greg,

thank you for your reply.

Just to clarify, with this proposal we are trying to address plugins in
non-default library path (I should have made up a better example, such
as "/opt/vendor/product/version/lib/plugins/$ISA/"). Default library
path already works fine for both 32 and 64-bit.

How about supporting the following macros, that would be settable at
configure with the following default values:

#define ISA_STRING "/$ISA/"
#define ISA_32 "/"
#define ISA_64 "/64/"

I believe this should offer enough flexibility.

Thanks in advance for comments.

Tomas
_______________________________________________
krbdev mailing list ***@mit.edu
https://mailman.mit.edu/mailman/listinfo/krbdev

[Benjamin Kaduk]

Post by Tomas Kuthan

Post by Greg Hudson

Post by Tomas Kuthan
We would like to solve that by supporting $ISA place holder in the path,
that would translate to '/64/' on 64-bit and to '/' on 32-bit. Hence the
module = pkinit:/lib/$ISA/site/preauth/pkinit.so
Would MIT be willing to accept a patch implementing something along
these lines?

My concern is that different operating systems have done different
things with 32-bit and 64-bit libraries, even within the space of Linux
distributions. Rather than a solution narrowly tailored to one
operating system, I would like a plan which can cover a broad range of
operating systems, preferrably one which doesn't add a lot of complexity
to the code.

Hi Greg,
thank you for your reply.
Just to clarify, with this proposal we are trying to address plugins in
non-default library path (I should have made up a better example, such
as "/opt/vendor/product/version/lib/plugins/$ISA/"). Default library
path already works fine for both 32 and 64-bit.
How about supporting the following macros, that would be settable at
#define ISA_STRING "/$ISA/"
#define ISA_32 "/"
#define ISA_64 "/64/"
I believe this should offer enough flexibility.

I'm unconvinced that 32- and 64-bit (x86) are the only ABIs that need to
be considered. ARM is an interesting case, with thumb-ABI as an option,
hard- or soft-float (not that we use much floating-point, but it's
illustrative of the ABI explosion), and other options once you start
permit running on different chips. It really seems like something more
generic than 32 vs 64 bits is going to be needed.

-Ben
_______________________________________________
krbdev mailing list ***@mit.edu
https://mailman.mit.edu/mailman/listinfo/krbdev

[Roland Mainz]

[snip]

Post by Benjamin Kaduk

Post by Tomas Kuthan
thank you for your reply.
Just to clarify, with this proposal we are trying to address plugins in
non-default library path (I should have made up a better example, such
as "/opt/vendor/product/version/lib/plugins/$ISA/"). Default library
path already works fine for both 32 and 64-bit.
How about supporting the following macros, that would be settable at
#define ISA_STRING "/$ISA/"
#define ISA_32 "/"
#define ISA_64 "/64/"
I believe this should offer enough flexibility.

I'm unconvinced that 32- and 64-bit (x86) are the only ABIs that need to
be considered. ARM is an interesting case, with thumb-ABI as an option,
hard- or soft-float (not that we use much floating-point, but it's
illustrative of the ABI explosion), and other options once you start
permit running on different chips. It really seems like something more
generic than 32 vs 64 bits is going to be needed.

The two "traditional" solutions we had at Sun were:
1. use some ${PATH}-like construct where the supported ISAs are listed
manually as ${PATH}-elements separated by ':' (the dynamic linker well
weed-out unsupported ISAs by refusing to load shared libraries those
ISA is not supported or incompatible)

XOR

2. Use the way listed in
http://src.illumos.org/source/xref/illumos-gate/usr/src/lib/libc/port/gen/isaexec.c
, which basically is a iteration/loop over all supported ISAs, uses a
base path, appends the current isa name and then tries to load/execute
code from that constructed path. If nothing is there the next ISA name
is tried.

[2] works fine on SystemV-derived UNIX (Solaris/Illumos, etc.)
/usr/lib contains soft-links with the ISA names to the "real"
directory location. This might work for LSB systems, too.

----

Bye,
Roland

[Tomas Kuthan]

Post by Benjamin Kaduk

Post by Tomas Kuthan

Post by Greg Hudson

Post by Tomas Kuthan
We would like to solve that by supporting $ISA place holder in the path,
that would translate to '/64/' on 64-bit and to '/' on 32-bit. Hence the
module = pkinit:/lib/$ISA/site/preauth/pkinit.so
Would MIT be willing to accept a patch implementing something along
these lines?

My concern is that different operating systems have done different
things with 32-bit and 64-bit libraries, even within the space of Linux
distributions. Rather than a solution narrowly tailored to one
operating system, I would like a plan which can cover a broad range of
operating systems, preferrably one which doesn't add a lot of complexity
to the code.

Hi Greg,
thank you for your reply.
Just to clarify, with this proposal we are trying to address plugins in
non-default library path (I should have made up a better example, such
as "/opt/vendor/product/version/lib/plugins/$ISA/"). Default library
path already works fine for both 32 and 64-bit.
How about supporting the following macros, that would be settable at
#define ISA_STRING "/$ISA/"
#define ISA_32 "/"
#define ISA_64 "/64/"
I believe this should offer enough flexibility.

I'm unconvinced that 32- and 64-bit (x86) are the only ABIs that need to
be considered. ARM is an interesting case, with thumb-ABI as an option,
hard- or soft-float (not that we use much floating-point, but it's
illustrative of the ABI explosion), and other options once you start
permit running on different chips. It really seems like something more
generic than 32 vs 64 bits is going to be needed.
-Ben

Hi Ben,

I am not familiar with thumb-ABI (basically all I know is that it exists).

So for example if I have an application compiled in thumb instruction
set and I try to dlopen a regular 32-bit ARM plugin, dlopen would fail?
Or can thumb and ARM code co-exist in this way?

Thanks,
Tomas
_______________________________________________
krbdev mailing list ***@mit.edu
https://mailman.mit.edu/mailman/listinfo/krbdev

[Benjamin Kaduk]

Post by Tomas Kuthan
I am not familiar with thumb-ABI (basically all I know is that it exists).
So for example if I have an application compiled in thumb instruction set and
I try to dlopen a regular 32-bit ARM plugin, dlopen would fail? Or can thumb
and ARM code co-exist in this way?

I'm not actually sure, since I don't do much work with ARM systems; it's
just something I remember reading about on the FreeBSD mailing lists.

-Ben
_______________________________________________
krbdev mailing list ***@mit.edu
https://mailman.mit.edu/mailman/listinfo/krbdev

[Russ Allbery]

Post by Tomas Kuthan
Just to clarify, with this proposal we are trying to address plugins in
non-default library path (I should have made up a better example, such
as "/opt/vendor/product/version/lib/plugins/$ISA/"). Default library
path already works fine for both 32 and 64-bit.
How about supporting the following macros, that would be settable at
#define ISA_STRING "/$ISA/"
#define ISA_32 "/"
#define ISA_64 "/64/"
I believe this should offer enough flexibility.

This wouldn't work for Debian and Debian-derived systems that support full
multiarch, not just the biarch system that Red Hat initially implemented.

The variation that probably would work for those systems would be to allow
the expansion of $ISA to be defined at compile time, so that on Debian
amd64 (for example) it would be defined as "/x86_64-linux-gnu/". But that
probably wouldn't be enough for Solaris, where you want to search multiple
paths.

[Tomas Kuthan]

Post by Russ Allbery

Post by Tomas Kuthan
Just to clarify, with this proposal we are trying to address plugins in
non-default library path (I should have made up a better example, such
as "/opt/vendor/product/version/lib/plugins/$ISA/"). Default library
path already works fine for both 32 and 64-bit.
How about supporting the following macros, that would be settable at
#define ISA_STRING "/$ISA/"
#define ISA_32 "/"
#define ISA_64 "/64/"
I believe this should offer enough flexibility.

This wouldn't work for Debian and Debian-derived systems that support full
multiarch, not just the biarch system that Red Hat initially implemented.
The variation that probably would work for those systems would be to allow
the expansion of $ISA to be defined at compile time, so that on Debian
amd64 (for example) it would be defined as "/x86_64-linux-gnu/". But that
probably wouldn't be enough for Solaris, where you want to search multiple
paths.

Hi Russ,

thank you for the pointer to the multiarch paradigm.
I guess that's back to the drawing board for me :-)

Tomas


_______________________________________________
krbdev mailing list ***@mit.edu
https://mailman.mit.edu/mailman/listinfo/krbdev