Richard Basch
2015-10-04 01:25:38 UTC
Out of curiosity, will the patches I provided last year against 1.13 for event-based iprop be included in 1.14?
I didn’t see it in the list of 1.14 features (http://k5wiki.kerberos.org/wiki/Release_1.14 <http://k5wiki.kerberos.org/wiki/Release_1.14>), though I submitted long prior to the feature cutoff deadline (Jun 2015) and provided similar patches since 1.12.
krbdev mailing list ***@mit.edu
https://mailman.mit.edu/mailma
I didn’t see it in the list of 1.14 features (http://k5wiki.kerberos.org/wiki/Release_1.14 <http://k5wiki.kerberos.org/wiki/Release_1.14>), though I submitted long prior to the feature cutoff deadline (Jun 2015) and provided similar patches since 1.12.
Subject: RE: krb5 1.13 Replication enhancements
Date: October 15, 2014 at 11:50:51 PM EDT
FYI. My “git diff” patches from beta1 applied cleanly into 1.13 “release”.
No surprise – I didn’t expect any code change between 1.13-beta1 and 1.13.
Hopefully, this can get rolled into 1.14.
Sent: Saturday, October 11, 2014 6:48 PM
Subject: RE: krb5 1.13 Replication enhancements
I found an issue with kdb_first_sno not being set during a DB restore which contributed to extra resyncs. Now, I believe all my prior 1.12 contributions are now covered by these patches.
Updated wiki: https://github.com/rbasch/krb5/wiki/1.13-Replication-enhancements <https://github.com/rbasch/krb5/wiki/1.13-Replication-enhancements>
https://github.com/rbasch/krb5/commit/d9ec51d87f9fc4e1592423bf7562895fad2752c2 <https://github.com/rbasch/krb5/commit/d9ec51d87f9fc4e1592423bf7562895fad2752c2> first_sno fix
https://github.com/rbasch/krb5/commit/f4ed3369f2c7a0ec6d834f46677d8ffc4e2223f4 <https://github.com/rbasch/krb5/commit/f4ed3369f2c7a0ec6d834f46677d8ffc4e2223f4> notify slaves
Sent: Wednesday, October 8, 2014 9:42 PM
Subject: RE: krb5 1.13 Replication enhancements
I found a problem in kproplog pursuant to my “first_sno fix”.
Revised wiki: https://github.com/rbasch/krb5/wiki/1.13-Replication-enhancements <https://github.com/rbasch/krb5/wiki/1.13-Replication-enhancements>
https://github.com/rbasch/krb5/commit/d3189af4fc65569881d6f553d8ab34f435794635 <https://github.com/rbasch/krb5/commit/d3189af4fc65569881d6f553d8ab34f435794635> first_sno fix
https://github.com/rbasch/krb5/commit/4fbe0f4b8f5aa771bc7173725f9303a3a5b62711 <https://github.com/rbasch/krb5/commit/4fbe0f4b8f5aa771bc7173725f9303a3a5b62711> notify slaves
Sent: Thursday, October 2, 2014 8:25 PM
Subject: RE: krb5 1.13 Replication enhancements
Revised commits (in particular, the second one only differs in the syslog output generated, not in the core functionality). I was concerned about excessive logging by kadmind’s servicing lots of slaves, so instead of indicating each slave being notified (which was useful during debugging), merely indicate the number of slaves being notified (being notified does not mean they “understand” the notification or will check-in).
https://github.com/rbasch/krb5/commit/ce238c4a19e117caedc7ce53f4780270b86d521c <https://github.com/rbasch/krb5/commit/ce238c4a19e117caedc7ce53f4780270b86d521c> first_sno fix
https://github.com/rbasch/krb5/commit/1bf5b212392ff2679250fcb3d3f197fee3ced3e1 <https://github.com/rbasch/krb5/commit/1bf5b212392ff2679250fcb3d3f197fee3ced3e1> notify slaves
I have also documented an alternate approach to implementing the feature in my Wiki, but it is far too involved for me to pursue. MIT should decide if they want to keep the separation of programs as it stands today, in which case this patch is probably among the better implementations possible.
Sent: Wednesday, October 1, 2014 9:22 PM
Subject: krb5 1.13 Replication enhancements
I have uploaded my commits to github (I am still testing, but I am reasonably in my git commit)
See https://github.com/rbasch/krb5/wiki/1.13-Replication-enhancements <https://github.com/rbasch/krb5/wiki/1.13-Replication-enhancements>
- Do not require the DB first_sno to be present in the ulog (this avoids extraneous db resyncs to downstream slaves)
- Implement downstream slave notification when ulog is updated.
The following are my GIT commits, but it may be better to reference the commits via the wiki in case I have to apply another fix and rebase.
Allow kdb_first_* not to be present in the ulog
https://github.com/rbasch/krb5/commit/ce238c4a19e117caedc7ce53f4780270b86d521c <https://github.com/rbasch/krb5/commit/ce238c4a19e117caedc7ce53f4780270b86d521c>
Notify downstream slaves of pending ulog updates
https://github.com/rbasch/krb5/commit/db0c450f6c09f4623bff3c3aff16794ba4e2c763 <https://github.com/rbasch/krb5/commit/db0c450f6c09f4623bff3c3aff16794ba4e2c763>
-----Original Message-----
Sent: Tuesday, September 30, 2014 3:26 PM
Subject: RE: krb5-1.13-beta1 iprop
Yeah, re #2... I originally went through all the code which I could find which pertained to the ulog parsing... the only other change I had was in kproplog at the time so that it wouldn't overrun a missing entry. I had added some defensive checks in my original code to only allow for 1 missing entry, but upon a ulog wrap, it would reset the number.
I'll try to re-incorporate all of this...
-----Original Message-----
Sent: Tuesday, September 30, 2014 2:58 PM
Subject: Re: krb5-1.13-beta1 iprop
See item #2 in
http://k5wiki.kerberos.org/wiki/Projects/Hierarchical_iprop#Related_problems <http://k5wiki.kerberos.org/wiki/Projects/Hierarchical_iprop#Related_problems>
Your implementation had a workaround for this issue, but it broke (at
least) the detection of ulog size changes. I ran out of time to design a better workaround.
Subject: RE: krb5 1.13 Replication enhancements
Date: October 15, 2014 at 11:50:51 PM EDT
FYI. My “git diff” patches from beta1 applied cleanly into 1.13 “release”.
No surprise – I didn’t expect any code change between 1.13-beta1 and 1.13.
Hopefully, this can get rolled into 1.14.
Sent: Saturday, October 11, 2014 6:48 PM
Subject: RE: krb5 1.13 Replication enhancements
I found an issue with kdb_first_sno not being set during a DB restore which contributed to extra resyncs. Now, I believe all my prior 1.12 contributions are now covered by these patches.
Updated wiki: https://github.com/rbasch/krb5/wiki/1.13-Replication-enhancements <https://github.com/rbasch/krb5/wiki/1.13-Replication-enhancements>
https://github.com/rbasch/krb5/commit/d9ec51d87f9fc4e1592423bf7562895fad2752c2 <https://github.com/rbasch/krb5/commit/d9ec51d87f9fc4e1592423bf7562895fad2752c2> first_sno fix
https://github.com/rbasch/krb5/commit/f4ed3369f2c7a0ec6d834f46677d8ffc4e2223f4 <https://github.com/rbasch/krb5/commit/f4ed3369f2c7a0ec6d834f46677d8ffc4e2223f4> notify slaves
Sent: Wednesday, October 8, 2014 9:42 PM
Subject: RE: krb5 1.13 Replication enhancements
I found a problem in kproplog pursuant to my “first_sno fix”.
Revised wiki: https://github.com/rbasch/krb5/wiki/1.13-Replication-enhancements <https://github.com/rbasch/krb5/wiki/1.13-Replication-enhancements>
https://github.com/rbasch/krb5/commit/d3189af4fc65569881d6f553d8ab34f435794635 <https://github.com/rbasch/krb5/commit/d3189af4fc65569881d6f553d8ab34f435794635> first_sno fix
https://github.com/rbasch/krb5/commit/4fbe0f4b8f5aa771bc7173725f9303a3a5b62711 <https://github.com/rbasch/krb5/commit/4fbe0f4b8f5aa771bc7173725f9303a3a5b62711> notify slaves
Sent: Thursday, October 2, 2014 8:25 PM
Subject: RE: krb5 1.13 Replication enhancements
Revised commits (in particular, the second one only differs in the syslog output generated, not in the core functionality). I was concerned about excessive logging by kadmind’s servicing lots of slaves, so instead of indicating each slave being notified (which was useful during debugging), merely indicate the number of slaves being notified (being notified does not mean they “understand” the notification or will check-in).
https://github.com/rbasch/krb5/commit/ce238c4a19e117caedc7ce53f4780270b86d521c <https://github.com/rbasch/krb5/commit/ce238c4a19e117caedc7ce53f4780270b86d521c> first_sno fix
https://github.com/rbasch/krb5/commit/1bf5b212392ff2679250fcb3d3f197fee3ced3e1 <https://github.com/rbasch/krb5/commit/1bf5b212392ff2679250fcb3d3f197fee3ced3e1> notify slaves
I have also documented an alternate approach to implementing the feature in my Wiki, but it is far too involved for me to pursue. MIT should decide if they want to keep the separation of programs as it stands today, in which case this patch is probably among the better implementations possible.
Sent: Wednesday, October 1, 2014 9:22 PM
Subject: krb5 1.13 Replication enhancements
I have uploaded my commits to github (I am still testing, but I am reasonably in my git commit)
See https://github.com/rbasch/krb5/wiki/1.13-Replication-enhancements <https://github.com/rbasch/krb5/wiki/1.13-Replication-enhancements>
- Do not require the DB first_sno to be present in the ulog (this avoids extraneous db resyncs to downstream slaves)
- Implement downstream slave notification when ulog is updated.
The following are my GIT commits, but it may be better to reference the commits via the wiki in case I have to apply another fix and rebase.
Allow kdb_first_* not to be present in the ulog
https://github.com/rbasch/krb5/commit/ce238c4a19e117caedc7ce53f4780270b86d521c <https://github.com/rbasch/krb5/commit/ce238c4a19e117caedc7ce53f4780270b86d521c>
Notify downstream slaves of pending ulog updates
https://github.com/rbasch/krb5/commit/db0c450f6c09f4623bff3c3aff16794ba4e2c763 <https://github.com/rbasch/krb5/commit/db0c450f6c09f4623bff3c3aff16794ba4e2c763>
-----Original Message-----
Sent: Tuesday, September 30, 2014 3:26 PM
Subject: RE: krb5-1.13-beta1 iprop
Yeah, re #2... I originally went through all the code which I could find which pertained to the ulog parsing... the only other change I had was in kproplog at the time so that it wouldn't overrun a missing entry. I had added some defensive checks in my original code to only allow for 1 missing entry, but upon a ulog wrap, it would reset the number.
I'll try to re-incorporate all of this...
-----Original Message-----
Sent: Tuesday, September 30, 2014 2:58 PM
Subject: Re: krb5-1.13-beta1 iprop
See item #2 in
http://k5wiki.kerberos.org/wiki/Projects/Hierarchical_iprop#Related_problems <http://k5wiki.kerberos.org/wiki/Projects/Hierarchical_iprop#Related_problems>
Your implementation had a workaround for this issue, but it broke (at
least) the detection of ulog size changes. I ran out of time to design a better workaround.
_______________________________________________Date: October 15, 2014 at 11:50:51 PM EDT
FYI. My “git diff” patches from beta1 applied cleanly into 1.13 “release”.
No surprise – I didn’t expect any code change between 1.13-beta1 and 1.13.
Hopefully, this can get rolled into 1.14.
Sent: Saturday, October 11, 2014 6:48 PM
Subject: RE: krb5 1.13 Replication enhancements
I found an issue with kdb_first_sno not being set during a DB restore which contributed to extra resyncs. Now, I believe all my prior 1.12 contributions are now covered by these patches.
Updated wiki: https://github.com/rbasch/krb5/wiki/1.13-Replication-enhancements <https://github.com/rbasch/krb5/wiki/1.13-Replication-enhancements>
https://github.com/rbasch/krb5/commit/d9ec51d87f9fc4e1592423bf7562895fad2752c2 <https://github.com/rbasch/krb5/commit/d9ec51d87f9fc4e1592423bf7562895fad2752c2> first_sno fix
https://github.com/rbasch/krb5/commit/f4ed3369f2c7a0ec6d834f46677d8ffc4e2223f4 <https://github.com/rbasch/krb5/commit/f4ed3369f2c7a0ec6d834f46677d8ffc4e2223f4> notify slaves
Sent: Wednesday, October 8, 2014 9:42 PM
Subject: RE: krb5 1.13 Replication enhancements
I found a problem in kproplog pursuant to my “first_sno fix”.
Revised wiki: https://github.com/rbasch/krb5/wiki/1.13-Replication-enhancements <https://github.com/rbasch/krb5/wiki/1.13-Replication-enhancements>
https://github.com/rbasch/krb5/commit/d3189af4fc65569881d6f553d8ab34f435794635 <https://github.com/rbasch/krb5/commit/d3189af4fc65569881d6f553d8ab34f435794635> first_sno fix
https://github.com/rbasch/krb5/commit/4fbe0f4b8f5aa771bc7173725f9303a3a5b62711 <https://github.com/rbasch/krb5/commit/4fbe0f4b8f5aa771bc7173725f9303a3a5b62711> notify slaves
Sent: Thursday, October 2, 2014 8:25 PM
Subject: RE: krb5 1.13 Replication enhancements
Revised commits (in particular, the second one only differs in the syslog output generated, not in the core functionality). I was concerned about excessive logging by kadmind’s servicing lots of slaves, so instead of indicating each slave being notified (which was useful during debugging), merely indicate the number of slaves being notified (being notified does not mean they “understand” the notification or will check-in).
https://github.com/rbasch/krb5/commit/ce238c4a19e117caedc7ce53f4780270b86d521c <https://github.com/rbasch/krb5/commit/ce238c4a19e117caedc7ce53f4780270b86d521c> first_sno fix
https://github.com/rbasch/krb5/commit/1bf5b212392ff2679250fcb3d3f197fee3ced3e1 <https://github.com/rbasch/krb5/commit/1bf5b212392ff2679250fcb3d3f197fee3ced3e1> notify slaves
I have also documented an alternate approach to implementing the feature in my Wiki, but it is far too involved for me to pursue. MIT should decide if they want to keep the separation of programs as it stands today, in which case this patch is probably among the better implementations possible.
Sent: Wednesday, October 1, 2014 9:22 PM
Subject: krb5 1.13 Replication enhancements
I have uploaded my commits to github (I am still testing, but I am reasonably in my git commit)
See https://github.com/rbasch/krb5/wiki/1.13-Replication-enhancements <https://github.com/rbasch/krb5/wiki/1.13-Replication-enhancements>
- Do not require the DB first_sno to be present in the ulog (this avoids extraneous db resyncs to downstream slaves)
- Implement downstream slave notification when ulog is updated.
The following are my GIT commits, but it may be better to reference the commits via the wiki in case I have to apply another fix and rebase.
Allow kdb_first_* not to be present in the ulog
https://github.com/rbasch/krb5/commit/ce238c4a19e117caedc7ce53f4780270b86d521c <https://github.com/rbasch/krb5/commit/ce238c4a19e117caedc7ce53f4780270b86d521c>
Notify downstream slaves of pending ulog updates
https://github.com/rbasch/krb5/commit/db0c450f6c09f4623bff3c3aff16794ba4e2c763 <https://github.com/rbasch/krb5/commit/db0c450f6c09f4623bff3c3aff16794ba4e2c763>
-----Original Message-----
Sent: Tuesday, September 30, 2014 3:26 PM
Subject: RE: krb5-1.13-beta1 iprop
Yeah, re #2... I originally went through all the code which I could find which pertained to the ulog parsing... the only other change I had was in kproplog at the time so that it wouldn't overrun a missing entry. I had added some defensive checks in my original code to only allow for 1 missing entry, but upon a ulog wrap, it would reset the number.
I'll try to re-incorporate all of this...
-----Original Message-----
Sent: Tuesday, September 30, 2014 2:58 PM
Subject: Re: krb5-1.13-beta1 iprop
If you want, I believe I can have patches for 1.13 beta in the next
couple days and publish them to github. In essence, there should be no
changes to command-line arguments at this point, since you have iprop
tree-based propagation already available.
Will that help?
Yes, that will provide a better starting point.couple days and publish them to github. In essence, there should be no
changes to command-line arguments at this point, since you have iprop
tree-based propagation already available.
Will that help?
Preliminary testing also suggests that it may be causing an extraneous
full resync (I had this problem in one of my early implementations
too). I suspect you are resetting the first_sno when you get your
first ulog entry (my 1.12 patches avoided resetting the first_sno
until it wrapped so that the first "dump/restore" would not be
followed by a second one if there were no additional ulog entries to
apply). I am not certain if this is what is going on, but some
preliminary testing suggested such (again, I haven't looked at the
1.13
code yet).full resync (I had this problem in one of my early implementations
too). I suspect you are resetting the first_sno when you get your
first ulog entry (my 1.12 patches avoided resetting the first_sno
until it wrapped so that the first "dump/restore" would not be
followed by a second one if there were no additional ulog entries to
apply). I am not certain if this is what is going on, but some
preliminary testing suggested such (again, I haven't looked at the
1.13
See item #2 in
http://k5wiki.kerberos.org/wiki/Projects/Hierarchical_iprop#Related_problems <http://k5wiki.kerberos.org/wiki/Projects/Hierarchical_iprop#Related_problems>
Your implementation had a workaround for this issue, but it broke (at
least) the detection of ulog size changes. I ran out of time to design a better workaround.
Subject: RE: krb5 1.13 Replication enhancements
Date: October 15, 2014 at 11:50:51 PM EDT
FYI. My “git diff” patches from beta1 applied cleanly into 1.13 “release”.
No surprise – I didn’t expect any code change between 1.13-beta1 and 1.13.
Hopefully, this can get rolled into 1.14.
Sent: Saturday, October 11, 2014 6:48 PM
Subject: RE: krb5 1.13 Replication enhancements
I found an issue with kdb_first_sno not being set during a DB restore which contributed to extra resyncs. Now, I believe all my prior 1.12 contributions are now covered by these patches.
Updated wiki: https://github.com/rbasch/krb5/wiki/1.13-Replication-enhancements <https://github.com/rbasch/krb5/wiki/1.13-Replication-enhancements>
https://github.com/rbasch/krb5/commit/d9ec51d87f9fc4e1592423bf7562895fad2752c2 <https://github.com/rbasch/krb5/commit/d9ec51d87f9fc4e1592423bf7562895fad2752c2> first_sno fix
https://github.com/rbasch/krb5/commit/f4ed3369f2c7a0ec6d834f46677d8ffc4e2223f4 <https://github.com/rbasch/krb5/commit/f4ed3369f2c7a0ec6d834f46677d8ffc4e2223f4> notify slaves
Sent: Wednesday, October 8, 2014 9:42 PM
Subject: RE: krb5 1.13 Replication enhancements
I found a problem in kproplog pursuant to my “first_sno fix”.
Revised wiki: https://github.com/rbasch/krb5/wiki/1.13-Replication-enhancements <https://github.com/rbasch/krb5/wiki/1.13-Replication-enhancements>
https://github.com/rbasch/krb5/commit/d3189af4fc65569881d6f553d8ab34f435794635 <https://github.com/rbasch/krb5/commit/d3189af4fc65569881d6f553d8ab34f435794635> first_sno fix
https://github.com/rbasch/krb5/commit/4fbe0f4b8f5aa771bc7173725f9303a3a5b62711 <https://github.com/rbasch/krb5/commit/4fbe0f4b8f5aa771bc7173725f9303a3a5b62711> notify slaves
Sent: Thursday, October 2, 2014 8:25 PM
Subject: RE: krb5 1.13 Replication enhancements
Revised commits (in particular, the second one only differs in the syslog output generated, not in the core functionality). I was concerned about excessive logging by kadmind’s servicing lots of slaves, so instead of indicating each slave being notified (which was useful during debugging), merely indicate the number of slaves being notified (being notified does not mean they “understand” the notification or will check-in).
https://github.com/rbasch/krb5/commit/ce238c4a19e117caedc7ce53f4780270b86d521c <https://github.com/rbasch/krb5/commit/ce238c4a19e117caedc7ce53f4780270b86d521c> first_sno fix
https://github.com/rbasch/krb5/commit/1bf5b212392ff2679250fcb3d3f197fee3ced3e1 <https://github.com/rbasch/krb5/commit/1bf5b212392ff2679250fcb3d3f197fee3ced3e1> notify slaves
I have also documented an alternate approach to implementing the feature in my Wiki, but it is far too involved for me to pursue. MIT should decide if they want to keep the separation of programs as it stands today, in which case this patch is probably among the better implementations possible.
Sent: Wednesday, October 1, 2014 9:22 PM
Subject: krb5 1.13 Replication enhancements
I have uploaded my commits to github (I am still testing, but I am reasonably in my git commit)
See https://github.com/rbasch/krb5/wiki/1.13-Replication-enhancements <https://github.com/rbasch/krb5/wiki/1.13-Replication-enhancements>
- Do not require the DB first_sno to be present in the ulog (this avoids extraneous db resyncs to downstream slaves)
- Implement downstream slave notification when ulog is updated.
The following are my GIT commits, but it may be better to reference the commits via the wiki in case I have to apply another fix and rebase.
Allow kdb_first_* not to be present in the ulog
https://github.com/rbasch/krb5/commit/ce238c4a19e117caedc7ce53f4780270b86d521c <https://github.com/rbasch/krb5/commit/ce238c4a19e117caedc7ce53f4780270b86d521c>
Notify downstream slaves of pending ulog updates
https://github.com/rbasch/krb5/commit/db0c450f6c09f4623bff3c3aff16794ba4e2c763 <https://github.com/rbasch/krb5/commit/db0c450f6c09f4623bff3c3aff16794ba4e2c763>
-----Original Message-----
Sent: Tuesday, September 30, 2014 3:26 PM
Subject: RE: krb5-1.13-beta1 iprop
Yeah, re #2... I originally went through all the code which I could find which pertained to the ulog parsing... the only other change I had was in kproplog at the time so that it wouldn't overrun a missing entry. I had added some defensive checks in my original code to only allow for 1 missing entry, but upon a ulog wrap, it would reset the number.
I'll try to re-incorporate all of this...
-----Original Message-----
Sent: Tuesday, September 30, 2014 2:58 PM
Subject: Re: krb5-1.13-beta1 iprop
If you want, I believe I can have patches for 1.13 beta in the next
couple days and publish them to github. In essence, there should be no
changes to command-line arguments at this point, since you have iprop
tree-based propagation already available.
Will that help?
Yes, that will provide a better starting point.couple days and publish them to github. In essence, there should be no
changes to command-line arguments at this point, since you have iprop
tree-based propagation already available.
Will that help?
Preliminary testing also suggests that it may be causing an extraneous
full resync (I had this problem in one of my early implementations
too). I suspect you are resetting the first_sno when you get your
first ulog entry (my 1.12 patches avoided resetting the first_sno
until it wrapped so that the first "dump/restore" would not be
followed by a second one if there were no additional ulog entries to
apply). I am not certain if this is what is going on, but some
preliminary testing suggested such (again, I haven't looked at the
1.13
code yet).full resync (I had this problem in one of my early implementations
too). I suspect you are resetting the first_sno when you get your
first ulog entry (my 1.12 patches avoided resetting the first_sno
until it wrapped so that the first "dump/restore" would not be
followed by a second one if there were no additional ulog entries to
apply). I am not certain if this is what is going on, but some
preliminary testing suggested such (again, I haven't looked at the
1.13
See item #2 in
http://k5wiki.kerberos.org/wiki/Projects/Hierarchical_iprop#Related_problems <http://k5wiki.kerberos.org/wiki/Projects/Hierarchical_iprop#Related_problems>
Your implementation had a workaround for this issue, but it broke (at
least) the detection of ulog size changes. I ran out of time to design a better workaround.
krbdev mailing list ***@mit.edu
https://mailman.mit.edu/mailma