From 978a3f4d0664c483dbcf93a176353e4cb5f93aea Mon Sep 17 00:00:00 2001 From: Jonathan Hui Date: Thu, 3 Jan 2019 18:54:49 +0000 Subject: [PATCH] Squashed 'third_party/nlbuild-autotools/repo/' changes from ac728423..75faaf5f 75faaf5f Addressed an issue in which the failure to make the 'repos-warning' target order-only caused dependent repo paths to be repeatedly and unnecessarily remade which, in turn, caused git operations and the configuration or build to fail. 1d4fff8c Merge pull request #16 from nestlabs/feature/repos-with-clone 7233d998 Added support for pulling down remote package dependencies using git clone (default) or git submodule depending on the setting of pull.method in repos.conf. 1debcf62 Added support for coverage-local and check-local targets. 4699860c Merge pull request #15 from nestlabs/feature/check-coverage-local 79923820 Added support for coverage-local and check-local targets. Added documentation for all coverage-related targets. d46f42a9 Addressed a spelling typo identified during code review. b6a6c1ac Take a different approach to version flapping against 'make distcheck' by leaving VERSION_FILE as an immediate (:=) variable and instead creating a two-level _VERSION variable, the first of which is set from VERSION_FILE by default at make recursion level zero (0). bf97ef6e Address an issue with VERSION_FILE on clean source code control clones or unarchived distribution in which 'make dist' or 'make distcheck' fail. VERSION_FILE should be and is intentionally a deferred (=) rather than an immediate (:=) variable to ensure late evaluation AFTER .local-version MAY be created rather than when the makefile containing it is parsed. ae96d615 Fixed a typo. 74c4da2c Added support to repos.conf for a 'commit' key such that a hash or tag other than HEAD for a given repository and branch may be synchronized and checked out. 0d8afe93 When the verbose flag is asserted, prnt out the actual series of action commands that will be invoked. 8663c890 * Address a number of issues with building the prepackaged GNU autotools subset included in nlbuild-autotools for Ubuntu Bionic. f2cdbac1 Add support, if necessary, for patching an expanded package. 35a238ed Only attempt to remove libtoolize if the effective user id matches that of the user executing the script. 7a899e8f Update the default version and change log for 1.5.2 and related changes. 48564b57 Since the '--name-only' option only appears in git-2.6 and later and there are production systems using git version as old as git-1.9, do not use '--name-only'. 7479acdf Fix a comment typo. 6c647b1d White space consistency clean-up. 60f989ea Incorporated a formatting suggestion from code review. 638a0c08 Fixed a typo. 33e0c7fc Addressed a number of typos and grammatical errors in comments and help output. 412d0e50 Fixed a typo. 4063134b Merge branch 'master' of github.com:nestlabs/nlbuild-autotools 4946bd54 Updated the date for 1.5.0. 90743059 Added a populate command for 'repos.conf'. 9bbf84f5 Added boilerplate examples for remote repository integration that package integrators can uncomment, adapt, or delete. 7418afd3 Initial revision. 0f9f1217 Ensure 'rm' explicitly includes -f should 'MAKEFLAGS' with '-R' be asserted in the environment. 2e6471f9 Add another clarifying example of building package-local tools from an integrated project. bad99a91 Added content for and updated default version to 1.5.0. b556c134 Added support for pulling down remote package dependencies using git submodule. b9c6ca41 Restore prior change introduced at b07c9c034bc039f41336f447b55a4640c466d2bc. e05e4fc4 Added a populate command to add 'Makefile-bootstrap' to the project top-level directory. 8ba67ce3 Added a populate command to add 'Makefile-bootstrap' to the project top-level directory. git-subtree-dir: third_party/nlbuild-autotools/repo git-subtree-split: 75faaf5f7b87cdd15c5d78ec61e8327e551a2a79 --- .default-version | 2 +- CHANGES | 83 ++++ Makefile | 20 +- README.md | 160 ++++++- automake/post/rules/coverage.am | 36 +- examples/Makefile-bootstrap | 106 +++++ examples/Makefile-tests.am | 4 +- examples/Makefile-third_party.am | 21 +- examples/Makefile-toplevel.am | 46 +- examples/configure.ac | 92 +++- examples/repos.conf | 31 ++ make/host/tools.mak | 24 + make/host/tools/bootstrap.mak | 23 + make/host/tools/tools.mak | 46 ++ make/post.mak | 23 + make/post/rules.mak | 24 + make/post/rules/bootstrap.mak | 126 +++++ make/post/rules/help.mak | 39 ++ make/post/rules/repos.mak | 228 +++++++++ make/pre.mak | 24 + make/pre/macros.mak | 25 + make/pre/macros/git.mak | 83 ++++ make/pre/macros/repos.mak | 34 ++ make/pre/macros/verbosity.mak | 72 +++ make/pre/tools.mak | 25 + scripts/bootstrap | 4 + scripts/mkskeleton | 4 +- .../automake.patches/automake-00.description | 10 + .../automake.patches/automake-00.patch | 12 + tools/packages/build | 108 ++++- .../packages/make/make.patches/make-00.patch | 442 ++++++++++++++++++ tools/packages/make/make.patches/make-00.url | 1 + tools/packages/make/make.url | 2 +- tools/packages/make/make.version | 2 +- 34 files changed, 1925 insertions(+), 57 deletions(-) create mode 100644 examples/Makefile-bootstrap create mode 100644 examples/repos.conf create mode 100644 make/host/tools.mak create mode 100644 make/host/tools/bootstrap.mak create mode 100644 make/host/tools/tools.mak create mode 100644 make/post.mak create mode 100644 make/post/rules.mak create mode 100644 make/post/rules/bootstrap.mak create mode 100644 make/post/rules/help.mak create mode 100644 make/post/rules/repos.mak create mode 100644 make/pre.mak create mode 100644 make/pre/macros.mak create mode 100644 make/pre/macros/git.mak create mode 100644 make/pre/macros/repos.mak create mode 100644 make/pre/macros/verbosity.mak create mode 100644 make/pre/tools.mak create mode 100644 tools/packages/automake/automake.patches/automake-00.description create mode 100644 tools/packages/automake/automake.patches/automake-00.patch create mode 100644 tools/packages/make/make.patches/make-00.patch create mode 100644 tools/packages/make/make.patches/make-00.url diff --git a/.default-version b/.default-version index 1c99cf0e8..9f05f9f2c 100644 --- a/.default-version +++ b/.default-version @@ -1 +1 @@ -1.4.4 +1.6.5 diff --git a/CHANGES b/CHANGES index f4a8ef51a..516c8f187 100644 --- a/CHANGES +++ b/CHANGES @@ -1,3 +1,86 @@ +1.6.5 (2018-11-19) + + * Addressed an issue in which the failure to make the 'repos-warning' + target order-only caused dependent repo paths to be repeatedly + and unnecessarily remade which, in turn, caused git operations + and the configuration or build to fail. + +1.6.4 (2018-11-14) + + * Added support for pulling down remote package dependencies using + git clone (default) or git submodule depending on the setting of + pull.method in repos.conf. + +1.6.3 (2018-10-19) + + * Added support for coverage-local and check-local targets. Added + documentation for all coverage-related targets. + +1.6.2 (2018-10-09) + + * Take a different approach to version flapping against 'make + distcheck' by leaving VERSION_FILE as an immediate (:=) variable + and instead creating a two-level _VERSION variable, the + first of which is set from VERSION_FILE by default at make + recursion level zero (0). + +1.6.1 (2018-09-26) + + * Address an issue with VERSION_FILE on clean source code control + clones or unarchived distribution in which 'make dist' or + 'make distcheck' fail. VERSION_FILE should be and is + intentionally a deferred (=) rather than an immediate (:=) + variable to ensure late evaluation AFTER .local-version MAY + be created rather than when the makefile containing it is + parsed. + +1.6.0 (2018-09-19) + + * Added support to repos.conf for a 'commit' key such that a + hash or tag other than HEAD for a given repository and + branch may be synchronized and checked out. + + * Made a minor change to the 'bootstrap' script such that it emits + the actual action commands that will be executed when the verbose + flag is asserted. + +1.5.3 (2018-09-14) + + * Address a number of issues with building the prepackaged GNU + autotools subset included in nlbuild-autotools for Ubuntu + Bionic. + + - Fixed "Unescaped left brace in regex is deprecated" that + was formerly a warning in Perl 5.22 and is now a hard error + in Perl 5.26. + + - Address new glibc glob interface changes by upgrading from + make-3.82 to make-4.1 and applying relevant Ubuntu Bionic + patches. + + * Add support to the package 'build' script to find and apply + patches for the above. + +1.5.2 (2018-04-26) + + * Since the '--name-only' option only appears in git-2.6 and later + and there are production systems using git version as old as + git-1.9, do not use '--name-only' when processing 'repos.conf'. + +1.5.1 (2018-04-20) + + * Addressed a number of typos and grammatical errors in comments + and help output. + +1.5.0 (2018-04-19) + + * Added support for pulling down remote package dependencies + using git submodule. + + * Addressed an issue in which nl_enable_coverage.m4 did not work + correctly on some Linux distributions by specifying coverage + libraries under LIBS rather than LDFLAGS. + 1.4.4 (2018-02-06) * Addressed an issue where 'mkskeleton' failed while trying to diff --git a/Makefile b/Makefile index 44e53f3f1..19ab6c7c9 100644 --- a/Makefile +++ b/Makefile @@ -1,5 +1,6 @@ # -# Copyright 2017 Nest Labs Inc. All Rights Reserved. +# Copyright (c) 2017-2018 Nest Labs Inc. All Rights Reserved. +# Copyright (c) 2018 Google LLC. All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. @@ -67,9 +68,24 @@ DISTFILES := $(shell $(CAT) MANIFEST) # .dist-version second, and the .default-version last. # +# VERSION_FILE should be and is intentionally an immediate (:=) rather +# than a deferred (=) variable to ensure the value binds once and only once +# for a given MAKELEVEL even as .local-version and .dist-version are created +# during makefile execution. + VERSION_FILE := $(if $(wildcard $(builddir)/.local-version),$(builddir)/.local-version,$(if $(wildcard $(srcdir)/.dist-version),$(srcdir)/.dist-version,$(srcdir)/.default-version)) -PACKAGE_VERSION ?= $(shell $(CAT) $(VERSION_FILE) 2> /dev/null) +# +# The two-level variables and the check against MAKELEVEL ensures that +# not only can the package version be overridden from the command line +# but also when the version is NOT overridden that we bind the version +# once and only once across potential sub-makes to prevent the version +# from flapping as VERSION_FILE changes. +# + +export MAYBE_PACKAGE_VERSION := $(if $(filter 0,$(MAKELEVEL)),$(shell cat $(VERSION_FILE) 2> /dev/null),$(MAYBE_PACKAGE_VERSION)) + +PACKAGE_VERSION ?= $(MAYBE_PACKAGE_VERSION) VERSION = $(PACKAGE_VERSION) diff --git a/README.md b/README.md index f09fd0ae6..08f969696 100644 --- a/README.md +++ b/README.md @@ -65,9 +65,15 @@ following POSIX-based build host systems: * x86_64-apple-darwin * x86_64-unknown-linux-gnu -### Build and Install +### Build and Install {#Build_and_Install} -Simply invoke `make tools` at the top-level of your nlbuild-autotools tree. +Simply invoke `make tools` at the top-level of your nlbuild-autotools +tree or, for example, from your package or project in which you have +integrated nlbuild-autotools: + +``` +make -C third_party/nlbuild-autotools/repo tools +``` ### Download and Expand @@ -103,12 +109,8 @@ The nlbuild-autotools package is laid out as follows: | automake/post.am | GNU automake Makefile.am footer included by every makefile. | | automake/pre/ | GNU automake Makefile.am headers. | | automake/pre.am | GNU automake Makefile.am header included by every makefile. | -| etc/ | Configurations files used by nlbuild-autotools. | | examples/ | Example template files for starting your own nlbuild-autotools-based project. | | scripts/ | Automation scripts for regenerating the build system and for managing package versions. | -| third_party/ | Third-party code and infrastructure used by nlbuild-autotools. | -| third_party/autoconf/ | GNU autoconf infrastructure provided by third-parties, including GNU. | -| third_party/autoconf/m4/ | GNU m4 macros for configure.ac provided by third-parties, including GNU. | | tools/ | Qualified packages of and pre-built instances of GNU autotools. | | tools/host/ | Pre-built instances of GNU autotools (if installed). | | tools/host/i686-pc-cygwin/ | Pre-built instances of GNU autotools for 32-bit Cygwin (if installed). | @@ -116,17 +118,121 @@ The nlbuild-autotools package is laid out as follows: | tools/host/x86_64-apple-darwin/ | Pre-built instances of GNU autotools for 64-bit Mac OS X (if installed). | | tools/host/x86_64-unknown-linux-gnu/ | Pre-built instances of GNU autotools for 64-bit Linux (if installed). | | tools/packages/ | Qualified packages for GNU autotools. | +## Internal Package Dependencies and Repositories -# Interact +Your package may have dependencies on other packages that can either +be inlined into your package or can be specified externally. If your +package has such dependencies, nlbuild-autotools contains support to +facilitate easy standalone tests and a successful 'make distcheck' +target (which effectively reqires 'configure' with no arguments to +produce a successful build) without incurring the costs of inlining +these dependencies into your own package. -There are numerous avenues for nlbuild-autotools support: +nlbuild-autotools supports this by providing a means to pull down +external git package repositories that your package depends on using +either git clone (default) or git submodules when you use and support +--with-=internal as a location for your dependent packages. - * Bugs and feature requests — [submit to the Issue Tracker](https://github.com/nestlabs/nlbuild-autotools/issues) - * Google Groups — discussion and announcements - * [nlbuild-autotools-announce](https://groups.google.com/forum/#!forum/nlbuild-autotools-announce) — release notes and new updates on nlbuild-autotools - * [nlbuild-autotools-users](https://groups.google.com/forum/#!forum/nlbuild-autotools-users) — discuss use of and enhancements to nlbuild-autotools +The example 'Makefile-bootstrap' has been provided as infrastructure to +make this easy for you as a package maintainer and for your package +users. The bootstrap makefile supports both the generic 'repos' target +to pull down all repositories on which your project depends as well as +relative path targets to where those repositories might live in your +project when they are internal (e.g, +third_party/package/repo). -# FAQ +Consequently, you can, for example, invoke: + +``` +% make -f Makefile-bootstrap repos +``` + +or + +``` +% make -f Makefile-bootstrap third_party/package/repo +``` + +to pull down all repositories or just the repository, for example, +placed at 'third_party/package/repos'. + +The bootstrap makefile generated for your package is yours to edit and +extend. In fact, hooks have been added so that you can do +package-specific work, including recursively pulling down repositories +for other packages. + +However, an even better and easier approach for your package users is +to integrate the bootstrap makefile repository process into your +configure script such that when an "internal" package location is +detected, it invokes the bootstrap makefile to perform this work on +behalf of the user. For example: + +``` +# Depending on whether my-package has been configured for an internal +# location, its directory stem within this package needs to be set +# accordingly. In addition, if the location is internal, then we need +# to attempt to pull it down using the bootstrap makefile. + +if test "${nl_with_my_package}" = "internal"; then + maybe_my_package_dirstem="my-package/repo" + my_package_dirstem="third_party/${maybe_my_package_dirstem}" + + AC_MSG_NOTICE([attempting to create internal ${my_package_dirstem}]) + ${MAKE-make} --no-print-directory -C ${ac_confdir} -f Makefile-bootstrap ${my_package_dirstem} + + if test $? -ne 0; then + AC_MSG_ERROR([failed to create ${my_package_dirstem}. Please check your network connection or the correctness of 'repos.conf']) + fi +else + maybe_my_package_dirstem="" +fi + +AC_SUBST(MY_PACKAGE_SUBDIRS, [${maybe_my_package_dirstem}]) +AM_CONDITIONAL([PACKAGE_WITH_MY_PACKAGE_INTERNAL], [test "${nl_with_my_package}" = "internal"]) +``` + +Note, the use of AC_SUBST on MY_PACKAGE_SUBDIRS to provide a mechanism +to conditionally populate SUBDIRS in the appropriate automake file +locations such that GNU autoconf does not generate syntax errors about +the potential absence of the subdirectory at bootstrap time. + +Of course, in either case, network connectivity is required to reach +the external git server hosting the packages on which your project +depends. + +In addition to the 'repos' target, the bootstrap makefile also +supports the 'clean-repos' target that undoes the work of the 'repos' +target. When using either the 'clone' or 'submodule' pull methods, it +will clean-up all of the synchronized repositories. When using the +'submodule' pull method, it also is careful to ensure it does not +disturb existing git or git submodule state your project might be +using. + +The infrastructure all works, of course, whether you are working in or +out of git and whether you have colocated or non-colocated source and +build directories. + +### Configuration + +This dependent repository feature of nlbuild-autotools uses a file, +'repos.conf', at the top level of your project source directory to +determine what external project repositories your package might want +to pull down, the location of their git server, the branch you want to +pull, and the location in your project in which you want to place +them. + +The format of 'repos.conf' _almost_ precisely follows that used by git +submodules with two notable additions, the 'pull' section and the +'commit' key. The 'pull' section allows you to optionally specify the +'method' key as 'clone' or 'submodule' (defaulting to 'clone' when the +key is not present). This selects whether 'git clone' or 'git +submodule' is used to pull down repositories. The 'commit' key allows +you to specify not only what branch to checkout but, more precisely, +what commit or tag to checkout rather than just _HEAD_ of a +branch. More information is available in 'Makefile-bootstrap' or with +`man gitmodules` or `git help gitmodules`. + +# FAQ {#FAQ} Q: Why does nlbuild-autotools have an option for its own built versions of GNU autotools rather than leveraging whatever versions exist on @@ -144,6 +250,34 @@ A: Some build host systems such as Mac OS X may not have GNU autotools standalone set of scripts to drive them without reliance on those versions of the tools on the build host system. +Q: When I rebootstrap my package, I see that a number of files related to + nlbuild-autotools have unexpectedly changed. Why is this happening? + +A: From time to time, the packages that comprise GNU autotools change + upstream. Frequently, common host operating systems (e.g., Ubuntu) take + a stable snapshot of the current autotools for a major release (e.g., + Ubuntu 14). On the next major release (e.g., Ubuntu 16), another snapshot + is taken. + + If your package was first bootstrapped with one version of + autotools and those bootstrap-generated files checked-in but you + later bootstrap with another version of autotools, then you will + likely observe this behavior. + + There are two solutions to this problem. First, to ensure a + consistent set of bootstrap-generated files, you can build the + autotools included with nlbuild-autotools. The bootstrap process + will always prefer to use those versions rather than those + available on the build host when they are available. See the + section [Build and Install](#Build_and_Install) for more + information. + + The second way to ensure a consistent set of bootstrap-generated + files is to not check them in. This does, however, require that + package users, rather than package maintainers, perform the + bootstrap process and does require that package users, rather than + package maintainers, have autotools available on the build host. + ## Maintainers If you are maintaining nlbuild-autotools, you have several key things to know: diff --git a/automake/post/rules/coverage.am b/automake/post/rules/coverage.am index aa54820f0..6410bb3f9 100644 --- a/automake/post/rules/coverage.am +++ b/automake/post/rules/coverage.am @@ -1,4 +1,5 @@ # +# Copyright 2018 Google LLC. All Rights Reserved. # Copyright 2015-2016 Nest Labs Inc. All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); @@ -26,9 +27,40 @@ # invocations. The 'check-am' and '$(BUILT_SOURCES)' are the key # automake-specific dependencies to ensure that happens. # +# This defines the following make targets: +# +# coverage: +# Execute local and recursive coverage targets across the +# project sub-tree at the current invocation point. +# +# This target is intended to be user-visibile and -invoked. +# +# coverage-local: +# Execute local coverage targets at the current invocation point. +# +# This target is intended to be user-visibile and -invoked. +# +# coverage-recursive: +# Recursively execute the 'coverage' target across the +# project sub-tree at the current invocation point. +# +# This target is NOT intended to be user-visibile and -invoked. +# +# check-local: +# Alias for the 'check-am' target. Execute local check +# targets at the current invocation point. Local coverage targets +# may depend on this to ensure local tests are made before +# coverage targets are made. +# +# This target is intended to be user-visibile and -invoked. +# + +.PHONY: coverage coverage-recursive coverage-local check-local + +check-local: check-am + +coverage: coverage-local -.PHONY: coverage coverage-recursive - coverage: coverage-recursive coverage: check-am diff --git a/examples/Makefile-bootstrap b/examples/Makefile-bootstrap new file mode 100644 index 000000000..592c35428 --- /dev/null +++ b/examples/Makefile-bootstrap @@ -0,0 +1,106 @@ +# +# Copyright 2017-2018 Nest Labs Inc. All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +# +# Description: +# This file is the makefile for inlining optional and required +# third-party packages as package-internal copies and for providing +# convenience targets for bootstrapping the GNU autotools-based build +# system used by this package. +# +# If your package has dependent git repositories that you might +# otherwise be inclined to subtree in, you can define a +# "repos.conf" file at the top of your project that enumerates +# the repositories, the branch thereof, and where to put it in +# your project. An example is shown below: +# +# [submodule "nlassert"] +# path = third_party/nlassert/repo +# url = git@github.com:nestlabs/nlassert.git +# branch = master +# update = none +# [submodule "nlunit-test"] +# path = third_party/nlunit-test/repo +# url = git@github.com:nestlabs/nlunit-test.git +# branch = master +# commit = 561a155bb7257e54198cf5496f74ed5bc41cf85f +# update = none +# + +ThisMakefile := $(firstword $(MAKEFILE_LIST)) + +builddir ?= . +abs_builddir ?= $(realpath $(builddir)) +top_builddir ?= $(builddir) +abs_top_builddir ?= $(realpath $(top_builddir)) +srcdir ?= $(dir $(realpath $(ThisMakefile))) +abs_srcdir ?= $(realpath $(srcdir)) +top_srcdir ?= $(srcdir) +abs_top_srcdir ?= $(realpath $(top_srcdir)) + +nlbuild_autotools_stem ?= @NLBUILD_AUTOTOOLS_STEM@ +abs_top_nlbuild_autotools_dir ?= $(abs_top_srcdir)/$(nlbuild_autotools_stem) + +include $(abs_top_nlbuild_autotools_dir)/make/pre.mak +include $(abs_top_nlbuild_autotools_dir)/make/host/tools/bootstrap.mak + +# +# Add any project-specific bootstrap help commands to this +# 'help-hook' target. +# +# These commands will be executed AFTER the core bootstrap 'help' +# target commands. +# +help-hook: + +# +# Add any project-specific bootstrap help commands to this +# 'help-bootstrap-hook' target. +# +# These commands will be executed AFTER the core bootstrap 'help' and +# 'help-bootstrap' target commands. +# +help-bootstrap-hook: + +# +# Add any project-specific bootstrap help commands to this +# 'help-repos-hook' target. +# +# These commands will be executed AFTER the core bootstrap 'help' and +# 'help-repos' target commands. +# +help-repos-hook: + +# +# Add any project-specific bootstrap repos commands to this +# 'repos-hook' target. +# +# These commands will be executed AFTER the core bootstrap 'repos' +# target commands. +# +repos-hook: + +# +# Add any project-specific bootstrap repos commands to this +# 'clean-repos-hook' target. +# +# These commands will be executed BEFORE the core bootstrap 'clean-repos' +# target commands. +# +clean-repos-hook: + +include $(abs_top_nlbuild_autotools_dir)/make/post.mak +include $(abs_top_nlbuild_autotools_dir)/make/post/rules/bootstrap.mak diff --git a/examples/Makefile-tests.am b/examples/Makefile-tests.am index f0529038e..19d50e32f 100644 --- a/examples/Makefile-tests.am +++ b/examples/Makefile-tests.am @@ -90,10 +90,10 @@ if @PACKAGE_SHORT_UPPER@_BUILD_COVERAGE_REPORTS $(@PACKAGE_SHORT_UPPER@_COVERAGE_BUNDLE): $(call create-directory) -$(@PACKAGE_SHORT_UPPER@_COVERAGE_INFO): check | $(@PACKAGE_SHORT_UPPER@_COVERAGE_BUNDLE) +$(@PACKAGE_SHORT_UPPER@_COVERAGE_INFO): check-local | $(@PACKAGE_SHORT_UPPER@_COVERAGE_BUNDLE) $(call generate-coverage-report,${top_builddir}) -coverage: $(@PACKAGE_SHORT_UPPER@_COVERAGE_INFO) +coverage-local: $(@PACKAGE_SHORT_UPPER@_COVERAGE_INFO) clean-local: clean-local-coverage diff --git a/examples/Makefile-third_party.am b/examples/Makefile-third_party.am index 519e61482..7bd0be595 100644 --- a/examples/Makefile-third_party.am +++ b/examples/Makefile-third_party.am @@ -27,9 +27,26 @@ include $(abs_top_nlbuild_autotools_dir)/automake/pre.am DIST_SUBDIRS = \ $(NULL) -# Always build (e.g. for 'make all') these subdirectories. +# Here is an example with nlunit-test. Uncomment and adapt or delete +# this, as needed. If you choose to use this, please also take a look at +# configure.ac and Makefile.am and uncomment the appropriate sections +# there. -SUBDIRS = \ +# # NLUNIT_TEST_SUBDIRS is not a permanent part of DIST_SUBDIRS since we do not +# # ever want to include it in a distribution archive; however, when it's been +# # pulled as a 'repo' module, we do want to remove it on invocation of the +# # 'distclean' target. Consequently, we conditionally include it in DIST_SUBDIRS +# # on invocation of 'distclean-recursive' +# +# distclean-recursive: DIST_SUBDIRS += $(NLUNIT_TEST_SUBDIRS) + +# Always build (e.g. for 'make all') these subdirectories. +# # +# # The value of NLUNIT_TEST_SUBDIRS will be populated by configure if +# # --with-nlunit_test=internal + +SUBDIRS = \ + $(NLUNIT_TEST_SUBDIRS) \ $(NULL) include $(abs_top_nlbuild_autotools_dir)/automake/post.am diff --git a/examples/Makefile-toplevel.am b/examples/Makefile-toplevel.am index e122b699e..5786e91ff 100644 --- a/examples/Makefile-toplevel.am +++ b/examples/Makefile-toplevel.am @@ -1,5 +1,6 @@ # -# Copyright 2016 Nest Labs Inc. All Rights Reserved. +# Copyright (c) 2016-2018 Nest Labs Inc. All Rights Reserved. +# Copyright (c) 2018 Google LLC. All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. @@ -31,6 +32,7 @@ SUBDIRS = \ $(NULL) EXTRA_DIST = \ + Makefile-bootstrap \ .default-version \ bootstrap \ bootstrap-configure \ @@ -68,6 +70,11 @@ DISTCLEANFILES = \ # .dist-version second, and the .default-version last. # +# VERSION_FILE should be and is intentionally an immediate (:=) rather +# than a deferred (=) variable to ensure the value binds once and only once +# for a given MAKELEVEL even as .local-version and .dist-version are created +# during makefile execution. + VERSION_FILE := $(if $(wildcard $(builddir)/.local-version),$(builddir)/.local-version,$(if $(wildcard $(srcdir)/.dist-version),$(srcdir)/.dist-version,$(srcdir)/.default-version)) # @@ -76,8 +83,16 @@ VERSION_FILE := $(if $(wildcard $(builddir)/.local-version) # version is always the current version, not the package bootstrap # version. # +# The two-level variables and the check against MAKELEVEL ensures that +# not only can the package version be overridden from the command line +# but also when the version is NOT overridden that we bind the version +# once and only once across potential sub-makes to prevent the version +# from flapping as VERSION_FILE changes. +# -@PACKAGE_SHORT_UPPER@_VERSION ?= $(shell cat $(VERSION_FILE) 2> /dev/null) +export MAYBE_@PACKAGE_SHORT_UPPER@_VERSION := $(if $(filter 0,$(MAKELEVEL)),$(shell cat $(VERSION_FILE) 2> /dev/null),$(MAYBE_@PACKAGE_SHORT_UPPER@_VERSION)) + +@PACKAGE_SHORT_UPPER@_VERSION ?= $(MAYBE_@PACKAGE_SHORT_UPPER@_VERSION) PACKAGE_VERSION = $(@PACKAGE_SHORT_UPPER@_VERSION) VERSION = $(PACKAGE_VERSION) @@ -136,10 +151,35 @@ $(distdir)/.dist-version: $(builddir)/.local-version force $(distdir)/.dist-version $(builddir)/.local-version: $(call check-file,$(@F)) -dist distcheck: $(BUILT_SOURCES) +# If you are synchronizing a package on which yours depends using 'repos.conf', +# to 'third_party', uncomment and adapt or delete this, as needed. If you choose +# to use this, please also take a look at Makefile.am and third_party/Makefile.am +# and uncomment the appropriate sections there. + +# # +# # When we run 'distcheck' and --with-nlunit_test defaults to 'internal', +# # the nlbuild-autotools infrastructure will attempt to create git paths +# # to manage the nlunit-test repo. Two directories need to be writable +# # to facilitate this. +# # +# DISTCHECK_CONFIGURE_FLAGS=`chmod u+w .. ../third_party` + +dist distcheck distdir install-headers: $(BUILT_SOURCES) dist-hook: $(distdir)/.dist-version +# If you are synchronizing a package on which yours depends using 'repos.conf', +# to 'third_party', uncomment and adapt or delete this, as needed. If you choose +# to use this, please also take a look at Makefile.am and third_party/Makefile.am +# and uncomment the appropriate sections there. + +# # +# # Ensure any locally synchronized repositories defined by 'repos.conf' +# # are cleaned up. +# # +# distclean-local: +# $(MAKE) -C $(srcdir) -f Makefile-bootstrap clean-repos + # # Top-level convenience target for making a documentation-only # distribution whose results appear at the top level of the build tree diff --git a/examples/configure.ac b/examples/configure.ac index 588a33113..02bc83d96 100644 --- a/examples/configure.ac +++ b/examples/configure.ac @@ -2,7 +2,7 @@ # Process this file with autoconf to produce a configure script. # -# Copyright 2016-2017 Nest Labs Inc. All Rights Reserved. +# Copyright 2016-2018 Nest Labs Inc. All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. @@ -31,7 +31,7 @@ AC_PREREQ([2.68]) # # Initialize autoconf for the package # -AC_INIT([@PACKAGE_SHORT_UPPER@], +AC_INIT([@PACKAGE_SHORT_LOWER@], m4_esyscmd([@NLBUILD_AUTOTOOLS_STEM@/scripts/mkversion -b `cat .default-version` .]), [@PACKAGE_EMAIL@], [@PACKAGE_SHORT_LOWER@], @@ -268,12 +268,69 @@ AM_CONDITIONAL(@PACKAGE_SHORT_UPPER@_BUILD_DOCS, [test "${nl_cv_build_docs}" = " # AC_MSG_NOTICE([checking required package dependencies]) -# NL_WITH_PACKAGE(...) - # Check if the build host has pkg-config AC_PATH_PROG([PKG_CONFIG],[pkg-config]) +# Here is a package example with nlunit-test. Uncomment and adapt or delete +# this, as needed. If you choose to use this, please also take a look at +# Makefile.am and third_party/Makefile.am and uncomment the appropriate +# sections there. + +# # +# # Nlunit-test +# # +# +# if test "${nl_cv_build_tests}" = "yes"; then +# NL_WITH_PACKAGE( +# [Nlunit-test], +# [NLUNIT_TEST], +# [nlunit_test], +# [-lnlunit-test], +# [ +# # At this point, the internal Nlunit-test package will be neither +# # configured nor built, so the normal checks we undertake for an +# # external package cannot be run here. Simply set the appropriate +# # variables and trust all will be well. +# +# NLUNIT_TEST_CPPFLAGS="-I\${abs_top_srcdir}/third_party/nlunit-test/repo/src" +# NLUNIT_TEST_LDFLAGS="-L${ac_pwd}/third_party/nlunit-test/repo/src" +# NLUNIT_TEST_LIBS="-lnlunit-test" +# ], +# [ +# # Check for required nlunit-test headers. +# +# AC_CHECK_HEADERS([nlunit-test.h], +# [], +# [ +# AC_MSG_ERROR(The nlunit-test header "$ac_header" is required but cannot be found.) +# ]) +# ]) +# fi +# +# # Depending on whether nlunit-test has been configured for an internal +# # location, its directory stem within this package needs to be set +# # accordingly. In addition, if the location is internal, then we need +# # to attempt to pull it down using the bootstrap makefile. +# +# if test "${nl_with_nlunit_test}" = "internal"; then +# maybe_nlunit_test_dirstem="nlunit-test/repo" +# nlunit_test_dirstem="third_party/${maybe_nlunit_test_dirstem}" +# +# AC_MSG_NOTICE([attempting to create internal ${nlunit_test_dirstem}]) +# +# ${MAKE-make} --no-print-directory -C ${srcdir} -f Makefile-bootstrap ${nlunit_test_dirstem} +# +# if test $? -ne 0; then +# AC_MSG_ERROR([failed to create ${nlunit_test_dirstem}. Please check your network connection or the correctness of 'repos.conf']) +# fi +# else +# maybe_nlunit_test_dirstem="" +# fi +# +# AC_SUBST(NLUNIT_TEST_SUBDIRS, [${maybe_nlunit_test_dirstem}]) +# AM_CONDITIONAL([@PACKAGE_SHORT_UPPER@_WITH_NLUNIT_TEST_INTERNAL], [test "${nl_with_nlunit_test}" = "internal"]) + # # Check for headers # @@ -303,17 +360,35 @@ if test "${ac_no_link}" != "yes"; then AC_CHECK_FUNCS([memcpy]) fi +# Here is an example with nlunit-test. Uncomment and adapt or delete +# this, as needed. + +# # Add any nlunit-test CPPFLAGS, LDFLAGS, and LIBS +# +# CPPFLAGS="${CPPFLAGS} ${NLUNIT_TEST_CPPFLAGS}" +# LDFLAGS="${LDFLAGS} ${NLUNIT_TEST_LDFLAGS}" +# LIBS="${LIBS} ${NLUNIT_TEST_LIBS}" + # Add any code coverage CPPFLAGS and LIBS CPPFLAGS="${CPPFLAGS} ${NL_COVERAGE_CPPFLAGS}" LIBS="${LIBS} ${NL_COVERAGE_LIBS}" # At this point, we can restore the compiler flags to whatever the -# user passed in, now that we're clear of an -Werror issues by +# user passed in, now that we're clear of any -Werror issues by # transforming -Wno-error back to -Werror. NL_RESTORE_WERROR +# Here is an example with nlunit-test. Uncomment and adapt or delete +# this, as needed. + +# # Configure any autotools-based subdirectories +# +# if test "${nl_with_nlunit_test}" = "internal"; then +# AC_CONFIG_SUBDIRS([third_party/nlunit-test/repo]) +# fi + # # Identify the various makefiles and auto-generated files for the package # @@ -362,6 +437,10 @@ AC_MSG_NOTICE([ Doxygen : ${DOXYGEN:--} GraphViz dot : ${DOT:--} PERL : ${PERL:--} + Nlunit-test source : ${nl_with_nlunit_test:--} + Nlunit-test compile flags : ${NLUNIT_TEST_CPPFLAGS:--} + Nlunit-test link flags : ${NLUNIT_TEST_LDFLAGS:--} + Nlunit-test link libraries : ${NLUNIT_TEST_LIBS:--} C Preprocessor : ${CPP} C Compiler : ${CC} C++ Preprocessor : ${CXXCPP} @@ -377,6 +456,3 @@ AC_MSG_NOTICE([ Link libraries : ${LIBS} ]) - - - diff --git a/examples/repos.conf b/examples/repos.conf new file mode 100644 index 000000000..dce15b772 --- /dev/null +++ b/examples/repos.conf @@ -0,0 +1,31 @@ +# You can synchronize a remote package on which yours depends with git +# to 'third_party' (or another directory). +# +# Here is an example using nlunit-test. Uncomment and adapt or +# delete this, as needed. If you do not have any packages to +# sychronize, then this file is unneeded and may be deleted from your +# project. +# +# If you choose to use this, please also take a look at configure.ac, +# Makefile.am, and third_party/Makefile.am and uncomment the appropriate +# sections there. +# +# The 'pull' section is optional. The 'pull' section allows you to +# optionally specify the 'method' key as 'clone' or 'submodule' +# (defaulting to 'clone' when the key is not present). This selects +# whether 'git clone' or 'git submodule' is used to pull down +# repositories. +# +# While the 'branch' key is required, the 'commit' key is +# optional. The 'commit' key allows you to specify a particular commit +# hash or tag if you do not want HEAD of the specified branch. + +# [pull] +# method = clone +# +# [submodule "nlunit-test"] +# path = third_party/nlunit-test/repo +# url = https://github.com/nestlabs/nlunit-test.git +# branch = master +# commit = 561a155bb7257e54198cf5496f74ed5bc41cf85f +# update = none diff --git a/make/host/tools.mak b/make/host/tools.mak new file mode 100644 index 000000000..04ee0f4fc --- /dev/null +++ b/make/host/tools.mak @@ -0,0 +1,24 @@ +# +# Copyright 2018 Nest Labs Inc. All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +# +# Description: +# This file is the generic "header" or pre make header for all +# common host-specific (i.e., non-target and -toolchain-specific) +# tools included in any makefile used in the build tree. +# + +include $(abs_top_nlbuild_autotools_dir)/make/host/tools/tools.mak diff --git a/make/host/tools/bootstrap.mak b/make/host/tools/bootstrap.mak new file mode 100644 index 000000000..5959df425 --- /dev/null +++ b/make/host/tools/bootstrap.mak @@ -0,0 +1,23 @@ +# +# Copyright 2018 Nest Labs Inc. All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +# +# Description: +# This file is the make header for the nlbuild-autotools +# bootstrap script. +# + +BOOTSTRAP ?= $(top_srcdir)/bootstrap diff --git a/make/host/tools/tools.mak b/make/host/tools/tools.mak new file mode 100644 index 000000000..2de99c933 --- /dev/null +++ b/make/host/tools/tools.mak @@ -0,0 +1,46 @@ +# +# Copyright 2018 Nest Labs Inc. All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +# +# Description: +# This file is the generic "header" or pre make header for all +# common host-specific (i.e., non-target and -toolchain-specific) +# tools included in any makefile used in the build tree. +# + +CAT ?= cat +CHMOD ?= chmod +CMP ?= cmp +CUT ?= cut +FIND ?= find +GIT ?= git +GREP ?= grep +GZIP ?= gzip +MKDIR ?= mkdir +MV ?= mv + +RMFLAGS = -f +ifeq ($(V),1) +RMFLAGS += -v +endif +RM ?= rm $(RMFLAGS) + +RMDIR ?= rmdir +SED ?= sed +SORT ?= sort +TAR ?= tar +UNIQ ?= uniq +XZ ?= xz diff --git a/make/post.mak b/make/post.mak new file mode 100644 index 000000000..702580b41 --- /dev/null +++ b/make/post.mak @@ -0,0 +1,23 @@ +# +# Copyright 2018 Nest Labs Inc. All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +# +# Description: +# This file is the generic "footer" or post make header included in +# any makefile used in the build tree. +# + +include $(abs_top_nlbuild_autotools_dir)/make/post/rules.mak diff --git a/make/post/rules.mak b/make/post/rules.mak new file mode 100644 index 000000000..da4503a53 --- /dev/null +++ b/make/post/rules.mak @@ -0,0 +1,24 @@ +# +# Copyright 2018 Nest Labs Inc. All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +# +# Description: +# This file is the generic "footer" or post make header for all +# common rules included in any makefile used in the build tree. +# + +include $(abs_top_nlbuild_autotools_dir)/make/post/rules/help.mak +include $(abs_top_nlbuild_autotools_dir)/make/post/rules/repos.mak diff --git a/make/post/rules/bootstrap.mak b/make/post/rules/bootstrap.mak new file mode 100644 index 000000000..36f07f662 --- /dev/null +++ b/make/post/rules/bootstrap.mak @@ -0,0 +1,126 @@ +# +# Copyright 2018 Nest Labs Inc. All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +# +# Description: +# This file is the make footer for nlbuild-autotools bootstrap +# convenience targets. +# + +# The following targets provide some convenience targets for bootstrapping +# the GNU autotools-based build system used by this package. + +all check coverage dist distcheck doc docdist install install-headers pretty pretty-check: Makefile + $(NL_V_MAKE)$(MAKE) -f $(<) --no-print-directory $(@) + +Makefile: $(top_srcdir)/Makefile.in $(top_srcdir)/configure + $(NL_V_CONFIGURE)$(top_srcdir)/configure + +$(top_srcdir)/configure: $(top_srcdir)/configure.ac + $(NL_V_BOOTSTRAP_CONFIG)$(BOOTSTRAP) -w config + +Makefile.in: Makefile.am + $(NL_V_BOOTSTRAP_MAKE)$(BOOTSTRAP) -w make + +.PHONY: bootstrap +bootstrap: + $(NL_V_BOOTSTRAP_ALL)$(BOOTSTRAP) -w all + +.PHONY: bootstrap-config +bootstrap-config: + $(NL_V_BOOTSTRAP_CONFIG)$(BOOTSTRAP) -w config + +.PHONY: bootstrap-make +bootstrap-make: + $(NL_V_BOOTSTRAP_MAKE)$(BOOTSTRAP) -w make + +define PrintBootstrapHelp +$(NL_V_AT)echo " all" +$(NL_V_AT)echo " Generate all configured build artifacts for this project." +$(NL_V_AT)echo +$(NL_V_AT)echo " bootstrap" +$(NL_V_AT)echo " (Re-)generate all build infrastructure for the project, " +$(NL_V_AT)echo " including both build configuration scripts and makefiles." +$(NL_V_AT)echo +$(NL_V_AT)echo " bootstrap-all" +$(NL_V_AT)echo " (Re-)generate all build infrastructure for the project, " +$(NL_V_AT)echo " including both build configuration scripts and makefiles." +$(NL_V_AT)echo +$(NL_V_AT)echo " bootstrap-config" +$(NL_V_AT)echo " (Re-)generate build configuration scripts for the project." +$(NL_V_AT)echo +$(NL_V_AT)echo " bootstrap-make" +$(NL_V_AT)echo " (Re-)generate build makefiles for the project." +$(NL_V_AT)echo +$(NL_V_AT)echo " check" +$(NL_V_AT)echo " Generate all configured build artifacts and run all unit " +$(NL_V_AT)echo " and functional tests for this project." +$(NL_V_AT)echo +$(NL_V_AT)echo " coverage" +$(NL_V_AT)echo " Generate all configured build artifacts, run all unit " +$(NL_V_AT)echo " and functional tests, and generate code coverage results " +$(NL_V_AT)echo " for this project." +$(NL_V_AT)echo +$(NL_V_AT)echo " dist" +$(NL_V_AT)echo " Generate an archive distribution snapshot for this project." +$(NL_V_AT)echo +$(NL_V_AT)echo " distcheck" +$(NL_V_AT)echo " Generate an archive distribution snapshot for this project " +$(NL_V_AT)echo " and sanity check the resulting distribution by running " +$(NL_V_AT)echo " 'make check' on it for this project." +$(NL_V_AT)echo +$(NL_V_AT)echo " doc" +$(NL_V_AT)echo " Generate documentation for the project." +$(NL_V_AT)echo +$(NL_V_AT)echo " docdist" +$(NL_V_AT)echo " Generate an archive distribution of the documentation for " +$(NL_V_AT)echo " the project." +$(NL_V_AT)echo +$(NL_V_AT)echo " install" +$(NL_V_AT)echo " Generate all configured build artifacts for this project " +$(NL_V_AT)echo " and install them in DESTDIR on the build host system." +$(NL_V_AT)echo +$(NL_V_AT)echo " install-headers" +$(NL_V_AT)echo " Generate all configured public header artifacts for this " +$(NL_V_AT)echo " project and install them in DESTDIR on the build host " +$(NL_V_AT)echo " system." +$(NL_V_AT)echo +$(NL_V_AT)echo " pretty" +$(NL_V_AT)echo " (Re-)format a collection of project source files." +$(NL_V_AT)echo +$(NL_V_AT)echo " pretty-check" +$(NL_V_AT)echo " Check but do not (re-)format a collection of project " +$(NL_V_AT)echo " source files." +$(NL_V_AT)echo +$(NL_V_AT)echo " Makefile" +$(NL_V_AT)echo " Run 'configure' for this project and generate the" +$(NL_V_AT)echo " host-specific makefile." +$(NL_V_AT)echo +endef # PrintBootstrapHelp + +.PHONY: help-bootstrap-local +help-bootstrap-local: + $(call PrintBootstrapHelp) + +.PHONY: help-bootstrap-hook +help-bootstrap-hook: help-bootstrap-local + +.PHONY: help-bootstrap +help-bootstrap: help-bootstrap-local help-bootstrap-hook + +.PHONY: help +help: help-bootstrap + diff --git a/make/post/rules/help.mak b/make/post/rules/help.mak new file mode 100644 index 000000000..8c9061535 --- /dev/null +++ b/make/post/rules/help.mak @@ -0,0 +1,39 @@ +# +# Copyright 2018 Nest Labs Inc. All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +# +# Description: +# This file is the make "footer" or post make header for help-related +# targets. +# + +define PrintHelp +$(NL_V_AT)echo "This makefile supports the following build targets:" +$(NL_V_AT)echo +$(NL_V_AT)echo " help" +$(NL_V_AT)echo " Display this content." +$(NL_V_AT)echo +endef # PrintHelp + +.PHONY: help-local +help-local: + $(call PrintHelp) + +.PHONY: help-hook +help-hook: help-local + +.PHONY: help +help: help-local help-hook diff --git a/make/post/rules/repos.mak b/make/post/rules/repos.mak new file mode 100644 index 000000000..51e4121db --- /dev/null +++ b/make/post/rules/repos.mak @@ -0,0 +1,228 @@ +# +# Copyright 2017-2018 Nest Labs Inc. All Rights Reserved. +# Copyright 2018 Google LLC. All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +# +# Description: +# This file is a make "footer" or post make header that defines make +# convenience targets and templates for interacting with and managing +# "foreign" (e.g., those outside of this project) git projects in the +# context of managing project dependencies. +# + +ifneq ($(REPOS),) +# Stem for git clones and submodules + +__REPOS_GIT_STEM := /.git + +# Stem for the git configuration file for a git repository. + +__REPOS_GIT_CONFIG_STEM := $(__REPOS_GIT_STEM)/config + +# Stem for the git cache directory for a git submodule + +__REPOS_GIT_MODULE_CACHE_STEM := $(__REPOS_GIT_STEM)/modules + +# git submodule configuration file and path + +__REPOS_GIT_MODULES_FILE := .gitmodules +__REPOS_GIT_MODULES_PATH := $(top_srcdir)/$(__REPOS_GIT_MODULES_FILE) + +# Stem for the git configuration file for a git clone or submodule. + +__REPOS_GIT_CLONE_STEM := $(__REPOS_GIT_STEM) +__REPOS_GIT_SUBMODULE_STEM := $(__REPOS_GIT_STEM) + +# Git "pull" method to use for retrieving repositories on which this +# package may depend. +# +# This defaults to 'clone' if no configuration value is present. + +__REPOS_MAYBE_PULL_METHOD := $(call nlGitGetMethodForPullFromFile,$(REPOS_CONFIG)) +__REPOS_DEFAULT_PULL_METHOD := clone +REPOS_PULL_METHOD := $(if $(__REPOS_MAYBE_PULL_METHOD),$(__REPOS_MAYBE_PULL_METHOD),$(__REPOS_DEFAULT_PULL_METHOD)) + +# Git repository configuration for this package, if it exists. +# +# This value is only relevant when the pull method is 'submodule'. + +ifeq ($(REPOS_PULL_METHOD),submodule) +REPOS_PACKAGE_GIT_PATH := $(top_srcdir)$(__REPOS_GIT_CONFIG_STEM) +else +REPOS_PACKAGE_GIT_PATH := +endif + +# Sentinel Files + +REPOS_GIT_INIT_SENTINEL := $(top_srcdir)/.repos-git-init-stamp +REPOS_GIT_MODULES_SENTINEL := $(top_srcdir)/.repos-git-modules-stamp +REPOS_WARNING_SENTINEL := $(top_builddir)/.repos-warning-stamp + +# +# REPOS_template +# +# This template defines variables and targets used for inlining optional and +# required third-party packages as package-internal copies. +# +# - Path to the repo configuration file from which to get +# values for named repo. +# - Name of the repository in for which to get +# values for branch, local path, and URL. +# +define REPOS_template +$(2)_repo_NAME := $(2) +$(2)_repo_BRANCH := $$(call nlGitGetBranchForRepoFromNameFromFile,$(1),$(2)) +$(2)_repo_COMMIT := $$(call nlGitGetCommitForRepoFromNameFromFile,$(1),$(2)) +$(2)_repo_PATH := $$(call nlGitGetPathForRepoFromNameFromFile,$(1),$(2)) +$(2)_repo_URL := $$(call nlGitGetURLForRepoFromNameFromFile,$(1),$(2)) + +$(2)_repo_GIT := $$(addsuffix $(__REPOS_GIT_STEM),$$($(2)_repo_PATH)) +$(2)_repo_CACHE := $(top_srcdir)$(__REPOS_GIT_MODULE_CACHE_STEM)/$$($(2)_repo_PATH) + +REPO_NAMES += $$($(2)_repo_NAME) +REPO_GITS += $$($(2)_repo_GIT) +REPO_PATHS += $$($(2)_repo_PATH) +REPO_URLS += $$($(2)_repo_URL) +REPO_CACHES += $$($(2)_repo_CACHE) + +# Allow a repo to be made with a path target (e.g., third_party/foo/repo) or +# with its actual git target (e.g., third_party/foo/repo/.git). + +$$($(2)_repo_PATH): | $$($(2)_repo_GIT) + +$$($(2)_repo_GIT): $(REPOS_PACKAGE_GIT_PATH) | repos-warning + $(NL_V_AT)case "$(REPOS_PULL_METHOD)" in \ + submodule) echo " SUBMODULE $$(subst $(__REPOS_GIT_SUBMODULE_STEM),,$$(@))"; \ + if ! test -f $(__REPOS_GIT_MODULES_PATH); then \ + touch $(REPOS_GIT_MODULES_SENTINEL); \ + fi; \ + $(GIT) -C $(top_srcdir) submodule -q add -f -b $$($(2)_repo_BRANCH) -- $$($(2)_repo_URL) $$($(2)_repo_PATH);; \ + clone) echo " CLONE $$(subst $(__REPOS_GIT_CLONE_STEM),,$$(@))"; \ + $(GIT) -C $(top_srcdir) clone -q -b $$($(2)_repo_BRANCH) -- $$($(2)_repo_URL) $$($(2)_repo_PATH);; \ + *) echo "$(REPOS_CONFIG): Unknown or unsupported pull method '$(REPOS_PULL_METHOD)'.";; \ + esac + $(NL_V_AT)if ! test -z "$$($(2)_repo_COMMIT)"; then \ + $(GIT) -C $$($(2)_repo_PATH) checkout -q $$($(2)_repo_COMMIT); \ + fi +endef # REPOS_template + +$(REPOS_PACKAGE_GIT_PATH): + $(NL_V_GIT_INIT)$(GIT) -C $(top_srcdir) init -q $(top_srcdir) + $(NL_V_AT)touch $(REPOS_GIT_INIT_SENTINEL) + +define PrintReposWarning +$(NL_V_AT)echo "The 'repos' target requires external network connectivity to" +$(NL_V_AT)echo "reach the following upstream GIT repositories:" +$(NL_V_AT)echo "" +$(NL_V_AT)for url in $(REPO_URLS); do echo " $${url}"; done +$(NL_V_AT)echo "" +$(NL_V_AT)echo "and will fail if external network connectivity is not" +$(NL_V_AT)echo "available. This package may still be buildable without these" +$(NL_V_AT)echo "packages but may require disabling certain features or" +$(NL_V_AT)echo "functionality." +$(NL_V_AT)echo "" +endef # PrintReposWarning + +$(REPOS_WARNING_SENTINEL): + $(NL_V_AT)touch $(@) + $(call PrintReposWarning) + +.PHONY: repos-warning +repos-warning: $(REPOS_WARNING_SENTINEL) + +.PHONY: repos-local +repos-local: repos-warning + $(NL_V_AT)$(MAKE) -f $(firstword $(MAKEFILE_LIST)) --no-print-directory $(REPO_GITS) + +.PHONY: repos-hook +repos-hook: repos-local + +.PHONY: repos +repos: repos-local repos-hook + +.PHONY: clean-repos-hook +clean-repos-hook: + +.PHONY: clean-repos-local +clean-repos-local: clean-repos-hook + @echo " CLEAN" + $(NL_V_AT)case "$(REPOS_PULL_METHOD)" in \ + submodule) $(GIT) -C $(top_srcdir) submodule -q deinit -f -- $(REPO_PATHS) 2> /dev/null || true; \ + if test -f $(REPOS_GIT_MODULES_SENTINEL); then \ + $(RM) $(REPOS_GIT_MODULES_SENTINEL); \ + $(GIT) -C $(top_srcdir) rm -f -q $(__REPOS_GIT_MODULES_PATH) 2> /dev/null; \ + fi ; \ + if test -f $(REPOS_GIT_INIT_SENTINEL); then \ + $(RM) -r $(dir $(REPOS_PACKAGE_GIT_PATH)); \ + $(RM) $(REPOS_GIT_INIT_SENTINEL); \ + fi; \ + $(RM) $(REPOS_WARNING_SENTINEL); \ + $(GIT) -C $(top_srcdir) rm -rf -q --cached $(REPO_PATHS) 2> /dev/null || true; \ + $(RM) -r $(addprefix $(top_srcdir)/,$(REPO_PATHS)); \ + $(RMDIR) -p $(addprefix $(top_srcdir),$(dir $(REPO_PATHS))) 2> /dev/null || true; \ + $(RM) -r $(REPO_CACHES) 2> /dev/null;; \ + clone) $(GIT) -C $(top_srcdir) rm -rf -q --cached $(REPO_PATHS) 2> /dev/null || true; \ + $(RM) $(REPOS_WARNING_SENTINEL); \ + $(RM) -r $(addprefix $(top_srcdir)/,$(REPO_PATHS)); \ + $(RMDIR) -p $(addprefix $(top_srcdir),$(dir $(REPO_PATHS))) 2> /dev/null || true;; \ + *) echo "$(REPOS_CONFIG): Unknown or unsupported pull method '$(REPOS_PULL_METHOD)'.";; \ + esac + +.PHONY: clean-repos +clean-repos: clean-repos-local + +# Invoke the REPOS_template for each defined optionally-inlined package repo + +$(foreach repo,$(REPOS),$(eval $(call REPOS_template,$(REPOS_CONFIG),$(repo)))) + +define MaybePrintReposHelp +$(NL_V_AT)echo " repos" +$(NL_V_AT)echo " Clone any upstream, dependent git repositories that this package" +$(NL_V_AT)echo " regards as required or optional rather than using '--with-'" +$(NL_V_AT)echo " options to specify external instances of those packages." +$(NL_V_AT)echo +$(NL_V_AT)echo " clean-repos" +$(NL_V_AT)echo " This is the opposite of the 'repos' target. This removes, in their" +$(NL_V_AT)echo " entirety, any clones of any upstream, dependent git repositories that" +$(NL_V_AT)echo " this package regards as required or optional." +$(NL_V_AT)echo +endef # MaybePrintReposHelp + +.DEFAULT_GOAL := repos +else + +define MaybePrintReposHelp +endef # MaybePrintReposHelp + +.DEFAULT_GOAL := help +endif # REPOS + +define PrintReposHelp +$(call MaybePrintReposHelp) +endef # PrintReposHelp + +.PHONY: help-repos-local +help-repos-local: + $(call PrintReposHelp) + +.PHONY: help-repos-hook +help-repos-hook: help-repos-local + +.PHONY: help-repos +help-repos: help-repos-local help-repos-hook + +.PHONY: help +help: help-repos diff --git a/make/pre.mak b/make/pre.mak new file mode 100644 index 000000000..62b204daa --- /dev/null +++ b/make/pre.mak @@ -0,0 +1,24 @@ +# +# Copyright 2018 Nest Labs Inc. All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +# +# Description: +# This file is the generic "header" or pre make header included in +# any makefile used in the build tree. +# + +include $(abs_top_nlbuild_autotools_dir)/make/pre/tools.mak +include $(abs_top_nlbuild_autotools_dir)/make/pre/macros.mak diff --git a/make/pre/macros.mak b/make/pre/macros.mak new file mode 100644 index 000000000..f92b4427a --- /dev/null +++ b/make/pre/macros.mak @@ -0,0 +1,25 @@ +# +# Copyright 2018 Nest Labs Inc. All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +# +# Description: +# This file is the make "header" or pre make header for macros common to +# all other make headers and files. +# + +include $(abs_top_nlbuild_autotools_dir)/make/pre/macros/git.mak +include $(abs_top_nlbuild_autotools_dir)/make/pre/macros/repos.mak +include $(abs_top_nlbuild_autotools_dir)/make/pre/macros/verbosity.mak diff --git a/make/pre/macros/git.mak b/make/pre/macros/git.mak new file mode 100644 index 000000000..f45907e0a --- /dev/null +++ b/make/pre/macros/git.mak @@ -0,0 +1,83 @@ +# +# Copyright 2018 Google LLC. All Rights Reserved. +# Copyright 2017-2018 Nest Labs Inc. All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +# +# Description: +# This file is the make "header" or pre make header that defines make +# convenience macros for interacting with git. +# + +# nlGitGetConfigFromFileCommand +# +# Command to get a value for a variable set in the specified git config file . + +nlGitGetConfigFromFileCommand = $(GIT) config --file $(1) + +# nlGitListConfigFromFileCommand +# +# Command to list all variables set in the specified config file + +nlGitListConfigFromFileCommand = $(call nlGitGetConfigFromFileCommand,$(1)) --list + +# nlGitGetConfigFromFile +# +# Get a value for a variable set in the specified git config file . + +nlGitGetConfigFromFile = $(shell $(call nlGitGetConfigFromFileCommand,$(1))) + +# nlGitListConfigFromFile +# +# List all variables set in the specified config file + +nlGitListConfigFromFile = $(shell $(call nlGitListConfigFromFileCommand,$(1))) + +# nlGitGetValueForRepoFromNameFromFile +# +# Get a value for a repo / submodule variable set in specified git config file . + +nlGitGetValueForRepoFromNameFromFile = $(shell $(call nlGitGetConfigFromFileCommand,$(1)) 'submodule.$(2).$(3)') + +# nlGitGetBranchForRepoFromNameFromFile +# +# Get the remote branch for a repo / submodule variable set in specified git config file . + +nlGitGetBranchForRepoFromNameFromFile = $(call nlGitGetValueForRepoFromNameFromFile,$(1),$(2),branch) + +# nlGitGetCommitForRepoFromNameFromFile +# +# Get the commit for a repo / submodule variable set in specified git config file . + +nlGitGetCommitForRepoFromNameFromFile = $(call nlGitGetValueForRepoFromNameFromFile,$(1),$(2),commit) + +# nlGitGetURLForRepoFromNameFromFile +# +# Get the remote URL for a repo / submodule variable set in specified git config file . + +nlGitGetURLForRepoFromNameFromFile = $(call nlGitGetValueForRepoFromNameFromFile,$(1),$(2),url) + +# nlGitGetPathForRepoFromNameFromFile +# +# Get the local path for a repo / submodule variable set in specified git config file . + +nlGitGetPathForRepoFromNameFromFile = $(call nlGitGetValueForRepoFromNameFromFile,$(1),$(2),path) + +# nlGitGetMethodForPullFromFile +# +# Get the pull method set in specified git config file . + +nlGitGetMethodForPullFromFile = $(shell $(call nlGitGetConfigFromFileCommand,$(1)) 'pull.method') + diff --git a/make/pre/macros/repos.mak b/make/pre/macros/repos.mak new file mode 100644 index 000000000..29f4b5169 --- /dev/null +++ b/make/pre/macros/repos.mak @@ -0,0 +1,34 @@ +# +# Copyright 2018 Google LLC. All Rights Reserved. +# Copyright 2017-2018 Nest Labs Inc. All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +# +# Description: +# This file is a make "header" or pre make header that defines +# make convenience macros for interacting with and managing "foreign" +# (e.g., those outside of this project) git projects in the context of +# managing project dependencies. +# + +# Third-party package repository configuration, if it exists, listing all +# packages and their names, branches, URLs, and local paths that may be +# optionally inlined for this package. +# +# This file is formatted identically to .gitmodules (see gitmodules(5)). + +REPOS_CONFIG ?= $(top_srcdir)/repos.conf + +REPOS := $(shell if test -f $(REPOS_CONFIG); then $(call nlGitListConfigFromFileCommand,$(REPOS_CONFIG)) | $(GREP) '^submodule\.' | $(CUT) -d '.' -f 2 | $(SORT) | $(UNIQ); fi) diff --git a/make/pre/macros/verbosity.mak b/make/pre/macros/verbosity.mak new file mode 100644 index 000000000..1fb5cb096 --- /dev/null +++ b/make/pre/macros/verbosity.mak @@ -0,0 +1,72 @@ +# +# Copyright 2017-2018 Nest Labs Inc. All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +# +# Description: +# This file is a make "header" or pre make header that defines make macros +# for controlling build verbosity. +# + +# +# Verbosity +# + +NL_DEFAULT_VERBOSITY ?= 0 + +NL_V_AT = $(NL_V_AT_$(V)) +NL_V_AT_ = $(NL_V_AT_$(NL_DEFAULT_VERBOSITY)) +NL_V_AT_0 = @ +NL_V_AT_1 = + +NL_V_BOOTSTRAP_ALL = $(NL_V_BOOTSTRAP_ALL_$(V)) +NL_V_BOOTSTRAP_ALL_ = $(NL_V_BOOTSTRAP_ALL_$(NL_DEFAULT_VERBOSITY)) +NL_V_BOOTSTRAP_ALL_0 = @echo " BOOTSTRAP all"; +NL_V_BOOTSTRAP_ALL_1 = + +NL_V_BOOTSTRAP_CONFIG = $(NL_V_BOOTSTRAP_CONFIG_$(V)) +NL_V_BOOTSTRAP_CONFIG_ = $(NL_V_BOOTSTRAP_CONFIG_$(NL_DEFAULT_VERBOSITY)) +NL_V_BOOTSTRAP_CONFIG_0 = @echo " BOOTSTRAP config"; +NL_V_BOOTSTRAP_CONFIG_1 = + +NL_V_BOOTSTRAP_MAKE = $(NL_V_BOOTSTRAP_MAKE_$(V)) +NL_V_BOOTSTRAP_MAKE_ = $(NL_V_BOOTSTRAP_MAKE_$(NL_DEFAULT_VERBOSITY)) +NL_V_BOOTSTRAP_MAKE_0 = @echo " BOOTSTRAP make"; +NL_V_BOOTSTRAP_MAKE_1 = + +NL_V_CONFIGURE = $(NL_V_CONFIGURE_$(V)) +NL_V_CONFIGURE_ = $(NL_V_CONFIGURE_$(NL_DEFAULT_VERBOSITY)) +NL_V_CONFIGURE_0 = @echo " CONFIGURE"; +NL_V_CONFIGURE_1 = + +NL_V_GIT_INIT = $(NL_V_GIT_INIT_$(V)) +NL_V_GIT_INIT_ = $(NL_V_GIT_INIT_$(NL_DEFAULT_VERBOSITY)) +NL_V_GIT_INIT_0 = @echo " GIT INIT $(@)"; +NL_V_GIT_INIT_1 = + +NL_V_MAKE = $(NL_V_MAKE_$(V)) +NL_V_MAKE_ = $(NL_V_MAKE_$(NL_DEFAULT_VERBOSITY)) +NL_V_MAKE_0 = @echo " MAKE $(@)"; +NL_V_MAKE_1 = + +NL_V_MKDIR_P = $(NL_V_MKDIR_P_$(V)) +NL_V_MKDIR_P_ = $(NL_V_MKDIR_P_$(NL_DEFAULT_VERBOSITY)) +NL_V_MKDIR_P_0 = @echo " MKDIR $(1)"; +NL_V_MKDIR_P_1 = + +NL_V_RMDIR = $(NL_V_RMDIR_$(V)) +NL_V_RMDIR_ = $(NL_V_RMDIR_$(NL_DEFAULT_VERBOSITY)) +NL_V_RMDIR_0 = @echo " RMDIR $(1)"; +NL_V_RMDIR_1 = diff --git a/make/pre/tools.mak b/make/pre/tools.mak new file mode 100644 index 000000000..96ae503bb --- /dev/null +++ b/make/pre/tools.mak @@ -0,0 +1,25 @@ +# +# Copyright 2018 Nest Labs Inc. All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +# +# Description: +# This file is the generic "header" or pre make header for all +# common (i.e., non-host-specific, non-target-specific, and +# non-toolchain-specific) tools included in any makefile used in the +# build tree. +# + +include $(abs_top_nlbuild_autotools_dir)/make/host/tools.mak diff --git a/scripts/bootstrap b/scripts/bootstrap index 352e25ec8..11aa6c884 100755 --- a/scripts/bootstrap +++ b/scripts/bootstrap @@ -277,6 +277,10 @@ esac # Bootstrap the package. +if [ -n "${verbose}" ]; then + echo "${local_action} && ${tool_action} && ${header_action} && ${make_action} && ${config_action}" +fi + ${local_action} && ${tool_action} && ${header_action} && ${make_action} && ${config_action} # Clean up any temporary files created. diff --git a/scripts/mkskeleton b/scripts/mkskeleton index 734904485..2a1233eda 100755 --- a/scripts/mkskeleton +++ b/scripts/mkskeleton @@ -1,7 +1,7 @@ #!/bin/bash # -# Copyright 2015-2016 Nest Labs Inc. All Rights Reserved. +# Copyright 2015-2018 Nest Labs Inc. All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. @@ -222,7 +222,9 @@ done populate "${DIR}" 664 configure.ac . populate "${DIR}" 775 bootstrap . +populate "${DIR}" 664 repos.conf . populate "${DIR}" 664 Makefile.am . +populate "${DIR}" 664 Makefile-bootstrap . populate "${DIR}" 664 Makefile.am doc populate "${DIR}" 664 Doxyfile.in doc populate "${DIR}" 664 Makefile.am third_party diff --git a/tools/packages/automake/automake.patches/automake-00.description b/tools/packages/automake/automake.patches/automake-00.description new file mode 100644 index 000000000..5523a74d4 --- /dev/null +++ b/tools/packages/automake/automake.patches/automake-00.description @@ -0,0 +1,10 @@ +2016-04-01 Paul Eggert + + automake: port to Perl 5.22 and later + + Without this change, Perl 5.22 complains "Unescaped left brace in + regex is deprecated" and this is planned to become a hard error in + Perl 5.26. See: + http://search.cpan.org/dist/perl-5.22.0/pod/perldelta.pod#A_literal_%22{%22_should_now_be_escaped_in_a_pattern + * bin/automake.in (substitute_ac_subst_variables): Escape left brace. + diff --git a/tools/packages/automake/automake.patches/automake-00.patch b/tools/packages/automake/automake.patches/automake-00.patch new file mode 100644 index 000000000..37707830a --- /dev/null +++ b/tools/packages/automake/automake.patches/automake-00.patch @@ -0,0 +1,12 @@ +diff -aruN a/bin/automake.in b/bin/automake.in +--- a/bin/automake.in 2013-11-02 17:48:13.000000000 -0700 ++++ b/bin/automake.in 2018-09-14 10:21:04.000000000 -0700 +@@ -3872,7 +3872,7 @@ + sub substitute_ac_subst_variables + { + my ($text) = @_; +- $text =~ s/\${([^ \t=:+{}]+)}/substitute_ac_subst_variables_worker ($1)/ge; ++ $text =~ s/\$[{]([^ \t=:+{}]+)}/substitute_ac_subst_variables_worker ($1)/ge; + return $text; + } + diff --git a/tools/packages/build b/tools/packages/build index a71c0c4c0..c464597f5 100755 --- a/tools/packages/build +++ b/tools/packages/build @@ -1,7 +1,8 @@ #!/bin/sh # -# Copyright 2014-2017 Nest Labs Inc. All Rights Reserved. +# Copyright 2014-2018 Nest Labs Inc. All Rights Reserved. +# Copyright 2018 Google LLC. All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. @@ -187,20 +188,20 @@ fetch_url_with_command() { shift 2 if [ -x "${executable}" ]; then - cd "${fetchdir}" - - verbose " `echo ${1} | tr '[[:lower:]]' '[[:upper:]]'` ${url}" + cd "${fetchdir}" + + verbose " `echo ${1} | tr '[[:lower:]]' '[[:upper:]]'` ${url}" - ${*} "${url}" + ${*} "${url}" - status=${?} + status=${?} - if [ ${?} -ne 0 ]; then - error "Failed to fetch ${url} with ${1}." - fi + if [ ${?} -ne 0 ]; then + error "Failed to fetch ${url} with ${1}." + fi - cd "${curdir}" - fi + cd "${curdir}" + fi return ${status} } @@ -264,26 +265,84 @@ fetch_package() { # script. # removetmp() { - rm -f "${LIBTOOLIZE}" + if [ -O "${LIBTOOLIZE}" ]; then + rm -f "${LIBTOOLIZE}" + fi if [ -n "${AUTOM4TE_CFG}" ]; then rm -f "${AUTOM4TE_CFG}" fi } # -# build_package +# patch_directory [ ... ] +# +# Patch the specified directory by applying the provided patched files +# using the specified patch arguments. The specified patch files may +# be uncompressed or compressed with any of bz2, gz, xz, compress, or +# zip. +# +patch_directory() { + local directory="${1}" + local patchargs="${2}" + + shift 2 + + verbose " PATCH ${directory}" + + for patch in "${*}"; do + verbose " PATCH ${patch}" + + extension=`echo "${patch}" | awk -F . '{if (NF > 1) {print $$NF}}'`; + + case "${extension}" in + + bz2) + uncompressor="bunzip2 -c" + ;; + + gz) + uncompressor="gunzip -c" + ;; + + xz) + uncompressor="xz -d -c" + ;; + + Z) + uncompressor="uncompress -c" + ;; + + zip) + uncompressor="unzip -p" + ;; + + *) + uncompressor="cat" + ;; + + esac + + ${uncompressor} "${patch}" | patch ${patchargs} -d "${directory}" || exit 1; + + done +} + +# +# build_package # # Build the specified package version for in from -# the archive found in and install it into the specified -# destination directory, . +# the archive found in , after applying the patches in +# (if any), and install it into the specified destination +# directory, . # build_package() { local package="${1}" local version="${2}" local host="${3}" - local archivedir="${4}" - local builddir="${5}" - local destdir="${6}" + local patchdir="${4}" + local archivedir="${5}" + local builddir="${6}" + local destdir="${7}" local fqpackage="${package}-${version}" local curdir=`pwd` local archive @@ -302,6 +361,12 @@ build_package() { tar --directory "${archivedir}" -xf "${archive}" || exit ${?} fi + # If necessary, patch the expanded package. + + if [ -d "${patchdir}" ]; then + patch_directory "${archivedir}/${fqpackage}" "-s -p1" ${patchdir}/*.patch* + fi + # If possible, attempt to be self-sufficient, relying on GNU autotools # executables installed along with the SDK itself. @@ -433,8 +498,8 @@ while [ ${#} -gt 0 ]; do ;; *) - usage 1 - ;; + usage 1 + ;; esac done @@ -481,6 +546,7 @@ banner "Building GNU autotools for ${ARCH}..." for package in ${PACKAGES}; do url="`cat ${SRCDIR}/${package}/${package}.url`" version="`cat ${SRCDIR}/${package}/${package}.version`" + patchdir="${SRCDIR}/${package}/${package}.patches" # Fetch, if necessary, the package from the canonical source location. @@ -494,7 +560,7 @@ for package in ${PACKAGES}; do mkdir -p "${BUILDDIR}/build/${ARCH}/${package}-${version}" || exit ${?} - build_package "${package}" "${version}" "${ARCH}" "${BUILDDIR}" "${BUILDDIR}/build/${ARCH}/${package}-${version}" "${DESTDIR}" + build_package "${package}" "${version}" "${ARCH}" "${patchdir}" "${BUILDDIR}" "${BUILDDIR}/build/${ARCH}/${package}-${version}" "${DESTDIR}" done trailer diff --git a/tools/packages/make/make.patches/make-00.patch b/tools/packages/make/make.patches/make-00.patch new file mode 100644 index 000000000..716faaf00 --- /dev/null +++ b/tools/packages/make/make.patches/make-00.patch @@ -0,0 +1,442 @@ +--- make-dfsg-4.1.orig/ar.c ++++ make-dfsg-4.1/ar.c +@@ -68,25 +68,39 @@ + + /* This function is called by 'ar_scan' to find which member to look at. */ + ++struct member_date_lookup ++{ ++ const char *name; ++ time_t *member_date; ++}; ++ + /* ARGSUSED */ + static long int + ar_member_date_1 (int desc UNUSED, const char *mem, int truncated, + long int hdrpos UNUSED, long int datapos UNUSED, + long int size UNUSED, long int date, + int uid UNUSED, int gid UNUSED, int mode UNUSED, +- const void *name) ++ const void *data) + { +- return ar_name_equal (name, mem, truncated) ? date : 0; ++ const struct member_date_lookup *lookup_data = data; ++ if (ar_name_equal (lookup_data->name, mem, truncated)) ++ { ++ *lookup_data->member_date = date; ++ return 1; ++ } ++ return 0; + } + +-/* Return the modtime of NAME. */ ++/* Read the modtime of NAME in MEMBER_DATE. ++ Returns 1 if NAME exists, 0 otherwise. */ + +-time_t +-ar_member_date (const char *name) ++int ++ar_member_date (const char *name, time_t *member_date) + { + char *arname; + char *memname; +- long int val; ++ int found; ++ struct member_date_lookup lookup_data; + + ar_parse_name (name, &arname, &memname); + +@@ -107,11 +121,14 @@ + (void) f_mtime (arfile, 0); + } + +- val = ar_scan (arname, ar_member_date_1, memname); ++ lookup_data.name = memname; ++ lookup_data.member_date = member_date; ++ found = ar_scan (arname, ar_member_date_1, &lookup_data); + + free (arname); + +- return (val <= 0 ? (time_t) -1 : (time_t) val); ++ /* return 0 (not found) if the archive does not exist or has invalid format. */ ++ return (found == 1) ? 1 : 0; + } + + /* Set the archive-member NAME's modtime to now. */ +--- make-dfsg-4.1.orig/commands.c ++++ make-dfsg-4.1/commands.c +@@ -622,7 +622,10 @@ + time_t file_date = (file->last_mtime == NONEXISTENT_MTIME + ? (time_t) -1 + : (time_t) FILE_TIMESTAMP_S (file->last_mtime)); +- if (ar_member_date (file->name) != file_date) ++ time_t member_date = NONEXISTENT_MTIME; ++ int found; ++ found = ar_member_date (file->name, &member_date); ++ if (found && member_date != file_date) + { + if (on_behalf_of) + OSS (error, NILF, +--- make-dfsg-4.1.orig/configure ++++ make-dfsg-4.1/configure +@@ -11060,10 +11061,9 @@ + #include + #include + +-#define GLOB_INTERFACE_VERSION 1 + #if !defined _LIBC && defined __GNU_LIBRARY__ && __GNU_LIBRARY__ > 1 + # include +-# if _GNU_GLOB_INTERFACE_VERSION == GLOB_INTERFACE_VERSION ++# if _GNU_GLOB_INTERFACE_VERSION == 1 || _GNU_GLOB_INTERFACE_VERSION == 2 + gnu glob + # endif + #endif], +--- make-dfsg-4.1.orig/dir.c ++++ make-dfsg-4.1/dir.c +@@ -748,8 +748,11 @@ + const char *slash; + + #ifndef NO_ARCHIVES +- if (ar_name (name)) +- return ar_member_date (name) != (time_t) -1; ++ { ++ time_t member_date; ++ if (ar_name (name)) ++ return ar_member_date (name, &member_date); ++ } + #endif + + #ifdef VMS +@@ -1212,15 +1215,40 @@ + } + #endif + ++/* Similarly for lstat. */ ++#if !defined(lstat) && !defined(WINDOWS32) || defined(VMS) ++# ifndef VMS ++# ifndef HAVE_SYS_STAT_H ++int lstat (const char *path, struct stat *sbuf); ++# endif ++# else ++ /* We are done with the fake lstat. Go back to the real lstat */ ++# ifdef lstat ++# undef lstat ++# endif ++# endif ++# define local_lstat lstat ++#elif defined(WINDOWS32) ++/* Windows doesn't support lstat(). */ ++# define local_lstat local_stat ++#else ++static int ++local_lstat (const char *path, struct stat *buf) ++{ ++ int e; ++ EINTRLOOP (e, lstat (path, buf)); ++ return e; ++} ++#endif ++ + void + dir_setup_glob (glob_t *gl) + { + gl->gl_opendir = open_dirstream; + gl->gl_readdir = read_dirstream; + gl->gl_closedir = free; ++ gl->gl_lstat = local_lstat; + gl->gl_stat = local_stat; +- /* We don't bother setting gl_lstat, since glob never calls it. +- The slot is only there for compatibility with 4.4 BSD. */ + } + + void +--- make-dfsg-4.1.orig/job.c ++++ make-dfsg-4.1/job.c +@@ -27,6 +27,14 @@ + + #include + ++#if defined (HAVE_LINUX_BINFMTS_H) && defined (HAVE_SYS_USER_H) ++#include ++#include ++#endif ++#ifndef PAGE_SIZE ++# define PAGE_SIZE (sysconf(_SC_PAGESIZE)) ++#endif ++ + /* Default shell to use. */ + #ifdef WINDOWS32 + #include +@@ -824,8 +832,6 @@ + break; + } + +- child_failed = exit_sig != 0 || exit_code != 0; +- + /* Search for a child matching the deceased one. */ + lastc = 0; + for (c = children; c != 0; lastc = c, c = c->next) +@@ -837,6 +843,15 @@ + Ignore it; it was inherited from our invoker. */ + continue; + ++ /* Determine the failure status: 0 for success, 1 for updating target in ++ question mode, 2 for anything else. */ ++ if (exit_sig == 0 && exit_code == 0) ++ child_failed = MAKE_SUCCESS; ++ else if (exit_sig == 0 && exit_code == 1 && question_flag && c->recursive) ++ child_failed = MAKE_TROUBLE; ++ else ++ child_failed = MAKE_FAILURE; ++ + DB (DB_JOBS, (child_failed + ? _("Reaping losing child %p PID %s %s\n") + : _("Reaping winning child %p PID %s %s\n"), +@@ -872,10 +887,10 @@ + delete non-precious targets, and abort. */ + static int delete_on_error = -1; + +- if (!dontcare) ++ if (!dontcare && child_failed == MAKE_FAILURE) + child_error (c, exit_code, exit_sig, coredump, 0); + +- c->file->update_status = us_failed; ++ c->file->update_status = child_failed == MAKE_FAILURE ? us_failed : us_question; + if (delete_on_error == -1) + { + struct file *f = lookup_file (".DELETE_ON_ERROR"); +@@ -987,7 +1002,7 @@ + if (!err && child_failed && !dontcare && !keep_going_flag && + /* fatal_error_signal will die with the right signal. */ + !handling_fatal_signal) +- die (MAKE_FAILURE); ++ die (child_failed); + + /* Only block for one child. */ + block = 0; +@@ -1189,14 +1204,15 @@ + ++p; + } + ++ child->recursive = ((flags & COMMANDS_RECURSE) != 0); ++ + /* Update the file's command flags with any new ones we found. We only + keep the COMMANDS_RECURSE setting. Even this isn't 100% correct; we are + now marking more commands recursive than should be in the case of + multiline define/endef scripts where only one line is marked "+". In + order to really fix this, we'll have to keep a lines_flags for every + actual line, after expansion. */ +- child->file->cmds->lines_flags[child->command_line - 1] +- |= flags & COMMANDS_RECURSE; ++ child->file->cmds->lines_flags[child->command_line - 1] |= flags & COMMANDS_RECURSE; + + /* POSIX requires that a recipe prefix after a backslash-newline should + be ignored. Remove it now so the output is correct. */ +@@ -3115,6 +3131,7 @@ + #ifdef WINDOWS32 + char *command_ptr = NULL; /* used for batch_mode_shell mode */ + #endif ++ char *args_ptr; + + # ifdef __EMX__ /* is this necessary? */ + if (!unixy_shell && shellflags) +@@ -3280,8 +3297,17 @@ + return new_argv; + } + ++#ifdef MAX_ARG_STRLEN ++ static char eval_line[] = "eval\\ \\\"set\\ x\\;\\ shift\\;\\ "; ++#define ARG_NUMBER_DIGITS 5 ++#define EVAL_LEN (sizeof(eval_line)-1 + shell_len + 4 \ ++ + (7 + ARG_NUMBER_DIGITS) * 2 * line_len / (MAX_ARG_STRLEN - 2)) ++#else ++#define EVAL_LEN 0 ++#endif ++ + new_line = xmalloc ((shell_len*2) + 1 + sflags_len + 1 +- + (line_len*2) + 1); ++ + (line_len*2) + 1 + EVAL_LEN); + ap = new_line; + /* Copy SHELL, escaping any characters special to the shell. If + we don't escape them, construct_command_argv_internal will +@@ -3301,6 +3327,30 @@ + #ifdef WINDOWS32 + command_ptr = ap; + #endif ++ ++#if !defined (WINDOWS32) && defined (MAX_ARG_STRLEN) ++ if (unixy_shell && line_len > MAX_ARG_STRLEN) ++ { ++ unsigned j; ++ memcpy (ap, eval_line, sizeof (eval_line) - 1); ++ ap += sizeof (eval_line) - 1; ++ for (j = 1; j <= 2 * line_len / (MAX_ARG_STRLEN - 2); j++) ++ ap += sprintf (ap, "\\$\\{%u\\}", j); ++ *ap++ = '\\'; ++ *ap++ = '"'; ++ *ap++ = ' '; ++ /* Copy only the first word of SHELL to $0. */ ++ for (p = shell; *p != '\0'; ++p) ++ { ++ if (isspace ((unsigned char)*p)) ++ break; ++ *ap++ = *p; ++ } ++ *ap++ = ' '; ++ } ++#endif ++ args_ptr = ap; ++ + for (p = line; *p != '\0'; ++p) + { + if (restp != NULL && *p == '\n') +@@ -3348,6 +3398,13 @@ + } + #endif + *ap++ = *p; ++#if !defined (WINDOWS32) && defined (MAX_ARG_STRLEN) ++ if (unixy_shell && line_len > MAX_ARG_STRLEN && (ap - args_ptr > MAX_ARG_STRLEN - 2)) ++ { ++ *ap++ = ' '; ++ args_ptr = ap; ++ } ++#endif + } + if (ap == new_line + shell_len + sflags_len + 2) + { +--- make-dfsg-4.1.orig/job.h ++++ make-dfsg-4.1/job.h +@@ -108,6 +108,7 @@ + unsigned int noerror:1; /* Nonzero if commands contained a '-'. */ + unsigned int good_stdin:1; /* Nonzero if this child has a good stdin. */ + unsigned int deleted:1; /* Nonzero if targets have been deleted. */ ++ unsigned int recursive:1; /* Nonzero for recursive command ('+' etc.) */ + unsigned int dontcare:1; /* Saved dontcare flag. */ + }; + +--- make-dfsg-4.1.orig/main.c ++++ make-dfsg-4.1/main.c +@@ -1401,13 +1401,18 @@ + #ifdef HAVE_ISATTY + if (isatty (fileno (stdout))) + if (! lookup_variable (STRING_SIZE_TUPLE ("MAKE_TERMOUT"))) +- define_variable_cname ("MAKE_TERMOUT", TTYNAME (fileno (stdout)), +- o_default, 0)->export = v_export; +- ++ { ++ const char *tty = TTYNAME (fileno (stdout)); ++ define_variable_cname ("MAKE_TERMOUT", tty ? tty : DEFAULT_TTYNAME, ++ o_default, 0)->export = v_export; ++ } + if (isatty (fileno (stderr))) + if (! lookup_variable (STRING_SIZE_TUPLE ("MAKE_TERMERR"))) +- define_variable_cname ("MAKE_TERMERR", TTYNAME (fileno (stderr)), +- o_default, 0)->export = v_export; ++ { ++ const char *tty = TTYNAME (fileno (stderr)); ++ define_variable_cname ("MAKE_TERMERR", tty ? tty : DEFAULT_TTYNAME, ++ o_default, 0)->export = v_export; ++ } + #endif + + /* Reset in case the switches changed our minds. */ +@@ -2418,6 +2423,11 @@ + exit (WIFEXITED(r) ? WEXITSTATUS(r) : EXIT_FAILURE); + } + #else ++#ifdef SET_STACK_SIZE ++ /* Reset limits, if necessary. */ ++ if (stack_limit.rlim_cur) ++ setrlimit (RLIMIT_STACK, &stack_limit); ++#endif + exec_command ((char **)nargv, environ); + #endif + free (aargv); +--- make-dfsg-4.1.orig/makeint.h ++++ make-dfsg-4.1/makeint.h +@@ -424,10 +424,11 @@ + /* The number of bytes needed to represent the largest integer as a string. */ + #define INTSTR_LENGTH CSTRLEN ("18446744073709551616") + ++#define DEFAULT_TTYNAME "true" + #ifdef HAVE_TTYNAME + # define TTYNAME(_f) ttyname (_f) + #else +-# define TTYNAME(_f) "true" ++# define TTYNAME(_f) DEFAULT_TTYNAME + #endif + + +@@ -478,7 +479,7 @@ + int ar_name (const char *); + void ar_parse_name (const char *, char **, char **); + int ar_touch (const char *); +-time_t ar_member_date (const char *); ++int ar_member_date (const char *, time_t *); + + typedef long int (*ar_member_func_t) (int desc, const char *mem, int truncated, + long int hdrpos, long int datapos, +--- make-dfsg-4.1.orig/remake.c ++++ make-dfsg-4.1/remake.c +@@ -1259,6 +1259,7 @@ + + char *arname, *memname; + struct file *arfile; ++ int found; + time_t member_date; + + /* Find the archive's name. */ +@@ -1306,10 +1307,15 @@ + /* The archive doesn't exist, so its members don't exist either. */ + return NONEXISTENT_MTIME; + +- member_date = ar_member_date (file->hname); +- mtime = (member_date == (time_t) -1 +- ? NONEXISTENT_MTIME +- : file_timestamp_cons (file->hname, member_date, 0)); ++ found = ar_member_date (file->hname, &member_date); ++ if (found && member_date == (time_t) 0) ++ { ++ OSS (error, NILF, ++ _("Warning: Archive '%s' seems to have been created in deterministic mode. '%s' will always be updated. Please consider passing the U flag to ar to avoid the problem."), ++ arfile->name, memname); ++ ++ } ++ mtime = found ? file_timestamp_cons (file->hname, member_date, 0) : NONEXISTENT_MTIME; + } + else + #endif +@@ -1548,9 +1554,11 @@ + { + static const char *dirs[] = + { ++#ifdef MULTIARCH_DIRS ++ MULTIARCH_DIRS ++#endif + #ifndef _AMIGA + "/lib", +- "/usr/lib", + #endif + #if defined(WINDOWS32) && !defined(LIBDIR) + /* +@@ -1559,7 +1567,19 @@ + */ + #define LIBDIR "." + #endif +- LIBDIR, /* Defined by configuration. */ ++ LIBDIR, /* Defined by configuration. */ ++#ifndef _AMIGA ++/* ++ * In the Debian binaries, PREFIX is /usr and thus this searches /lib, ++ * /usr/lib and /usr/lib again and therefore misses any libraries that ++ * are not packaged and were installed by the site admin. The ideal ++ * behaviour would be to have the search path set by a Makefile ++ * variable (other than the VPATH blunt object) but even absent that, ++ * it would be more useful if it looked in /usr/local/lib even though ++ * make itself hasn't been installed in the /usr/local tree -- manoj ++ */ ++ "/usr/local/lib", ++#endif + 0 + }; + diff --git a/tools/packages/make/make.patches/make-00.url b/tools/packages/make/make.patches/make-00.url new file mode 100644 index 000000000..26bd50103 --- /dev/null +++ b/tools/packages/make/make.patches/make-00.url @@ -0,0 +1 @@ +http://archive.ubuntu.com/ubuntu/pool/main/m/make-dfsg/make-dfsg_4.1-9.1ubuntu1.diff.gz diff --git a/tools/packages/make/make.url b/tools/packages/make/make.url index 729ec98e7..08e287025 100644 --- a/tools/packages/make/make.url +++ b/tools/packages/make/make.url @@ -1 +1 @@ -ftp://ftp.gnu.org/pub/gnu/make/make-3.82.tar.gz +ftp://ftp.gnu.org/pub/gnu/make/make-4.1.tar.gz diff --git a/tools/packages/make/make.version b/tools/packages/make/make.version index 818a025a4..7d5c902e7 100644 --- a/tools/packages/make/make.version +++ b/tools/packages/make/make.version @@ -1 +1 @@ -3.82 +4.1