= Proposal for "Do the Right Thing" Alternative Selection, by Dave Abrahams = [[PageOutline(1-6, Contents)]] This page describes a proposal for a change to the way target alternatives are selected. == Motivation == It seems there are a significant number of {{{bjam}}} invocations for which there is a sensible interpretation, but for which Boost.Build currently generates an error instead and refuses to build. === Example 1 === Given the following Jamfile: {{{ lib a : a.cpp : off ; # alternative 1 lib a : a-dbg.cpp : on on ; # alternative 2 }}} In response to {{{ bjam debug-symbols=on }}} the result is {{{ error: No best alternative for ./a next alternative: required properties: off not matched next alternative: required properties: on on not matched }}} My claim is that it would be more appropriate to build the 2nd alternative because it is a better match for the explicit build request. Boost.Build generates an error because the default value of the {{{}}} feature is "{{{off}}}". Boost.Build combines that default value with the explicit build request to get {{{on off}}}, which does not match the 2nd alternative. === Example 2 === I know at this point you're probably saying "the error makes sense; you asked for something for which there's no alternative with matching requirements." However, "no matching requirements" is currently ''not'' an error when there's only one alternative declared: {{{ # b and c both have only one alternative. lib b : b.cpp : off ; lib c : c.cpp : on on ; }}} in this case, {{{bjam debug-symbols=on}}} happily builds both {{{b}}} (with debug-symbols off) and {{{c}}} (with profiling on) despite the fact that the requirements for {{{b}}} directly contradict the explicit build request and that the requirements for {{{c}}} contradicts the default for {{{profiling}}}, which is implicitly {{{off}}}. The behavior in example 2 may seem counterintuitive, but it is widely regarded as desirable to succeed in building ''something'', even if it doesn't exactly match what the user asked for, than to cause an error. I'm not challenging that premise in this proposal. In fact, there's an easy way to understand this behavior: '''''requirements'' describe how a target must be built, not exactly how it must be requested.''' One point of this proposal is to make the case where there are multiple alternatives more consistent with that principle. === Definition of "Best" === Indeed, the error message didn't say "there's no matching alternative;" it said there's "no ''best'' alternative." Whether or not we accept this proposal turns on how we define "best." I propose that in example 1, the fact that alternative 2 matches an ''explicitly specified'' build property should make it "better" than alternative 1. === Example 3 === This example is a simplified, but equivalent, version of the example in ticket #16: {{{ lib test : : test gcc multi ; lib test : : test.lib msvc multi ; }}} a simple "{{{bjam toolset=gcc}}}" currently yields: {{{ error: No best alternative for ./test next alternative: required properties: multi gcc not matched next alternative: required properties: multi msvc not matched }}} Here Boost.Build complains because the {{{single}}} is the default. If you read ticket #16, you can see that the intended result was that Boost.Build choose the first alternative. My proposed change would accomplish that with no change in the Jamfile. We would lose the capability of specifying these alternatives in such a way that the build request {{{bjam toolset=gcc}}} causes an error. === Example 4 === This is a real-life example that resulted in a long [http://thread.gmane.org/gmane.comp.lib.boost.testing/4202/focus=4266 thread] on the boost-testing mailing list, and was finally resolved only through an intensive back-and-forth with Martin over IRC. I should add that it was this difficult to debug ''despite the fact that I had already written most of this proposal''; I just failed to recognize this as a manifestation of the same problem. Martin's test-config.jam file is shown at the bottom of http://thread.gmane.org/gmane.comp.lib.boost.testing/4202/focus=4271. The crucial fragment is: {{{ using python : 2.3 : /usr : : ; ... using gcc : 3.4.5_linux : /usr/local/gcc-3.4.5/bin/g++wrap --limit-memory=600 --limit-cpu=1800 : /usr/local/gcc-3.4.5 off ; using python : 2.4 : /usr/local/python/2.4/gcc-3.4.4 : : : gcc 3.4.5_linux ; }}} Here, Martin configures python 2.3 as the default, and then configures a specific build of python 2.4 to be used with gcc-3.4.5_linux. Then he runs his tests, in the simplest case, with: {{{ bjam gcc-3.4.5_linux }}} and the system uses the wrong python (2.3) to build and run these tests. To understand the reasons, you must know that {{{using python : }}} ''version'' {{{ : ... : : : }}} ''condition'' {{{;}}} ends up declaring a python library with the following requirements: ''condition'' {{{}}}''version'' As a result of the first {{{using python}}}... invocation, the default value of {{{}}} is "{{{2.3}}}". Since "{{{python=}}}..." was not given explicitly in the build request, the current semantics add the default value to the request and thus fail to match the intended python installation. In fact, to test with multiple toolsets and their intended versions of Python, Martin has to invoke {{{bjam}}} as follows: {{{bjam gcc-3.3.6_linux gcc-3.4.5_linux/python=2.4 gcc-4.0.2_linux/python=2.4}}} ''etc...'' This is not only inconvenient, but counterintuitive and uses a syntax that is not widely known (is it even documented?). Under my proposal, the intended version of Python would've been selected. == Why Not Separate Alternative Selection Criteria from Requirements? == One might try to address these cases by forcing the user to specify their criteria for selecting alternatives separately from the alternatives' build requirements. That would be a bad idea, for two reasons. First, ''the user usually wants a target to be built with the properties he explicitly requested, and often isn't aware of the default values of features that he didn't request.'' Currently, the explicitly-requested features are treated exactly like the defaulted ones, but the information about which ones were explicitly-requested is valuable; there's no reason to throw it out just because occasionally one ''additionally'' wants properties that aren't in the build request. Certainly, when the build request turns out to match an alternative's requirements exactly, Boost.Build currently considers that to be unambiguous, and that has not caused any real difficulty. This proposal expands the cases that Boost.Build considers unambiguous to better leverage the correspondence between explicit requests and desired requirements. Secondly, with alternative selection criteria completely separated from requirements, to express the same thing as one could express with two alternatives in this proposal: lib foo : foo.cpp ; lib foo : bar.cpp : value,,1,, value,,2,, ... value,,N,, ; would take N! (yes, that's N factorial) alternatives. '''Note:''' I concede that there are a few interesting cases one can't express today because Boost.Build doesn't allow one to separately specify alternative selection criteria, like an alternative that must be built with intel C++ when the requested toolset is msvc. However, these cases are at best very unusual; if they do need to be accomodated, a separate feature is warranted, and there's no reason not to make the usual cases work concisely. == Details of Proposed Semantics == The semantics I'm proposing depend on the notion of "explicitly-requested" build properties. '''Definition:''' a top-level target's explicitly-requested build properties are those that are specified on the command line, e.g. {{{bjam }}} '''{{{threading=multi}}}'''. A dependency target's explicitly-requested build properties are those that are propagated from its dependent target(s). As a special case, {{{}}} is always considered to be explicitly requested, even when not explicitly specified on the command line. When choosing among target alternatives, the one with the greatest number of requirements matching its explicitly-requested properties is selected. An ambiguity error is only reported when two alternatives match the same number of requirements. When searching for dependency targets, the dependent's requirements are combined with its explicitly-requested build properties to form a new set of explicitly-requested properties, to be applied to its dependencies. During combination, where a non-free feature in the dependent's requirements conflicts with its explicitly-requested properties, the requirement is selected over the the explicitly-requested value of that feature. == Not A Pure Extension == This is almost a pure extension (i.e. one that does not change the semantics of any currently-working cases) but not quite. One case that it would change is as follows ('''Example 3'''): {{{ lib a : a.cpp ; # 1st alternative lib a : a-dbg.cpp : multi on ; # 2nd alternative }}} Today, {{{ bjam debug-symbols=on }}} will select the first alternative. Under my proposal, it would select the 2nd. That leaves these questions: 1. Is that an important difference? That is for you to answer. 1. How would we get the current semantics? Read on... == Optional Extension to this Proposal == Currently, sa target alternative (when it's not the only alternative) will only be selected when ''all'' its requirements are matched in the build request. It's not yet clear that the rule "match all requirements" is actually useful in real-world scenarios (I admit that not matching requirements is a bit weird, but remember the rule of thumb: ''requirements describe how a target must be built, not exactly how it must be requested''). If we decide it is important to support alternatives whose requirements must ''all'' be present in the build request in order to be selected, we would need a new notation. I suggest: {{{ lib a : a.cpp ; # 1st alternative lib a : a-dbg.cpp : multi/on ; # 2nd alternative }}} In this case, the 2nd alternative would only be selected if ''both'' {{{multi}}} and {{{on}}} were in the explicitly-requested properties. It would probably make sense to weight matches to these alternatives more heavily. A reasonable way of scoring that accounts for these [http://www.boost.org/regression-logs/cs-win32_metacomm/doc/html/bbv2/reference.html#bbv2.reference.features property paths] might be: Sum( for all matching property paths ''p'', 10^''s''(''p'')^ ) where ''s''(''p'') :== the number of slashes in ''p'' This syntax would also be more consistent with the way compound build requests are specified on the command line: {{{ bjam toolset=gcc-3.4.5/debug-symbols=on toolset=intel/optimization=off }}} == Summary == This proposal would make several problematic cases into non-problems, would make the handling of multiple alternatives more uniform with the handling of "single alternatives," ''could'' make the syntax/semantics of specifying conditions for alternatives more consistent with that of build requests, and would make some currently-"ambiguous" bjam invocations into well-defined commands. Admittedly, it would change the semantics of a common class of target declarations when combined with what appears to be an uncommon class of bjam invocations (so uncommon, I claim, that the changes are probably not important to anyone). So far, we don't have a use case for the current semantics that isn't equally well-covered by this proposal. However, should we find such a use case, the "optional extension" detailed here would allow us to recover the current semantics (with a different target declaration syntax, of course).