#build-harness (2023-04)

Help with the Cloud Posse build-harness https://github.com/cloudposse/build-harness

2023-04-03

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

unsure if this is the right place, but I’m seeing this locally and on github actions randomly.

Cloning <https://github.com/cloudposse/build-harness.git#master>...
Cloning into 'build-harness'...
make[1]: Entering directory '/home/runner/work/infrastructure/infrastructure'
.build-harness:1: *** missing separator.  Stop.
David Schmidt avatar
David Schmidt

We often see this sort of failure when it’s really a failure to bootstrap - which in turn is often either rate-limiting, or not supplying good creds to not get rate-limited.

unsure if this is the right place, but I’m seeing this locally and on github actions randomly.

Cloning <https://github.com/cloudposse/build-harness.git#master>...
Cloning into 'build-harness'...
make[1]: Entering directory '/home/runner/work/infrastructure/infrastructure'
.build-harness:1: *** missing separator.  Stop.
johncblandii avatar
johncblandii

it could be rate limiting since it is random

David Schmidt avatar
David Schmidt

We always supply user creds so we get the benefit of our subscription rather than hoping it goes through

1
David Schmidt avatar
David Schmidt

A typical Makefile that bootstraps starts like this (we use -extensions extensively) :

# Bootstrap (pull) the build harness

# GITHUB_USER containing '@' char must be escaped with '%40'
GITHUB_USER := $(shell echo $(GITHUB_USER) | sed 's/@/%40/g')
GITHUB_TOKEN ?=

BUILD_HARNESS_EXTENSIONS_BRANCH=master

-include $(shell [ -f ".build-harness-bootstrap" ] || curl -sL -o .build-harness-bootstrap -H "Authorization: token $(GITHUB_TOKEN)" -H "Accept: application/vnd.github.v3.raw" "<https://raw.github.com/open-cluster-management/build-harness-extensions/master/templates/Makefile.build-harness-bootstrap>"; echo .build-harness-bootstrap)
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Interesting…. @Jeremy G (Cloud Posse), should we consider adding something like @David Schmidt’s answer?

Jeremy G (Cloud Posse) avatar
Jeremy G (Cloud Posse)

@johncblandii There was some breakage to build-harness overnight, because build-harness pulls in packages which has had a few bugs introduced as we enhanced it to support packaging for arm64, but nothing that I saw like “missing separator”. There were also 2 GitHub incidents that might have contributed to the error:

  1. Incident with Git Operations
  2. Incident with Actions and Pages
    So I expect this was a transient error, but if it happens again, please see if you can track down where the missing separator actually is.

@Erik Osterman (Cloud Posse) We curl the bootstrap file from <https://raw.githubusercontent.com> which

  1. Has a generous rate limit of 5000 request per IP address per hour
  2. Does not accept authentication and therefore does not benefit from it After that, we use git clone to download the repo, which automatically uses whatever user credentials are available. @David Schmidt Have you hit rate limits on [raw.githubusercontent.com](http://raw.githubusercontent.com)?
Incident with Git Operations

GitHub’s Status Page - Incident with Git Operations.

Incident with Actions and Pages

GitHub’s Status Page - Incident with Actions and Pages.

1
David Schmidt avatar
David Schmidt

We only pull build harness / build harness extensions with our own orgs, so I can’t speak to that.

David Schmidt avatar
David Schmidt

So if, for example, anyone else forks and uses them locally (in their own org), they will hit this problem if they use build harness with any regularity.

Jeremy G (Cloud Posse) avatar
Jeremy G (Cloud Posse)
04:37:46 PM

@Jeremy G (Cloud Posse) has joined the channel

johncblandii avatar
johncblandii

did something change with make packages/install/terraform-docs?

johncblandii avatar
johncblandii

yeah, something is off.

johncblandii avatar
johncblandii

oh…logs tell me directly:

Package terraform-docs no longer supported. Skipping

Jeremy G (Cloud Posse) avatar
Jeremy G (Cloud Posse)

@johncblandii Where did you see this error? terraform-docs is still supported by Cloud Posse, now on both amd64 and arm64.

johncblandii avatar
johncblandii

in our gha run

johncblandii avatar
johncblandii

let me add it back. 1 sec

johncblandii avatar
johncblandii
      - name: Install terraform-docs
        shell: bash
        env:
          INSTALL_PATH: "${{ github.workspace }}/bin"
        run: |
          make init
          mkdir -p "${INSTALL_PATH}"
          make packages/install/terraform-docs
          echo "$INSTALL_PATH" >> $GITHUB_PATH
Jeremy G (Cloud Posse) avatar
Jeremy G (Cloud Posse)

As I said, packages had some bugs introduced with the arm64 support, but it should all be fixed now. I just ran make packages/install/terraform-docs and it worked for me.

Jeremy G (Cloud Posse) avatar
Jeremy G (Cloud Posse)

Right, that was a bug

• introduced in packages PR 3503 at 2023-04-02T2327Z and

• fixed in PR 3506 at 2023-04-03T2112Z

johncblandii avatar
johncblandii

TF docs passes now and pre-commit ran

1
Jeremy G (Cloud Posse) avatar
Jeremy G (Cloud Posse)

@Erik Osterman (Cloud Posse) @Andriy Knysh (Cloud Posse) FYI, for about 22 hours yesterday, make packages/install was broken and either errored out (if run with /bin/sh) or claimed the package was no longer supported. Anyone reporting such errors during that time period should simply be advised to try it again.

2
    keyboard_arrow_up