#codefresh (2018-12)

@michal.matyjek has joined the channel

Anyone is posting back from Codefresh to github? Looking for a complete solution to post back to github PR (not to commit) with previous comment deletion/removal. Ideally with some templating built-in

Use case: post helm diff, or twistlock scan results to PR - in a way there’s only one comment for latest commit there.

@michal.matyjek yes, we have our github-commenter for that

https://github.com/cloudposse/github-commenter

it supports templated comments

does it overwrite?

no - that would be a “feature request”

if someone on the gladly team wants to contribute that, we’ll promptly review and get it merged

seems like it would be a nice feature, to be able to target a previous comment using some kind of regex

also getting the PR number from commit sha…. maybe using https://platform.github.community/t/find-pull-request-containing-specific-commit-hash/2827/6

--upsert-comment="/some regex/"

@Daren ^^

@Daren has joined the channel

getting the PR number from commit sha.

why?

We use the CF_PR_NUMBER or something

oh is that in place?

i had no idea

yea, sec

I think it did not always work for us or something - have not checked in months though

CF_PULL_REQUEST_NUMBER

we use it like this NAMESPACE=pr-${{CF_PULL_REQUEST_NUMBER}}-${{CF_REPO_NAME}}

If you trigger a build manually, then you don’t get it

oh yeah - I think it’s only if your trigger is on PR?

it’s only available on synchronize

or if you pass it manually

if you trigger on commit - you also dont I think

but that’s maybe a tell-tale sign you don’t want to execute

comment on PR - so that you can upsert previous commits

since it’s happening outside of a PR

you don’t want to do anything.

well if you push new commits to PR you do want it….

that’s synchronize

that’s supported by codefresh

hmmm… ok thanks!

@Andriy Knysh (Cloud Posse) can you scope out this feature? see it it would be pretty easy to implement?

sure

override prev comment is what you are after, @michal.matyjek?

yea, but not just any previous comment

the previous related comment

so maybe use a regex input?

yeah

and it would be like an “upsert”

update or if not found insert

can be some magic string, since we can have multiple steps commenting to github from Codefresh - each comment can have separate magic string

ok let me see what could be done

the other option @Daren mentioned was

I like how k8s repos replace the comment
they actually delete, then recomment

@michal.matyjek do you have a preference?

I do not, it’s OK to delete/recreate. Noone reads email here anyways except me and I am fine handling additonal github email notifications

I like the delete/repost. Then the comment is inline with the action that triggered it

Updating an existing comment can be confusing on PR with a lot of activity

Yea, good point

@Andriy Knysh (Cloud Posse)

Since you can post html as the body, you might be able to use metadata tags as a means of labeling the comment for deletion

Interesting, I looked a helm chart PR to see if I could spot how k8s does their automated comments. They are not posted as review comments, they show up as Issue comments

I dont know the difference

@Erik Osterman (Cloud Posse) how do you handle posting helm diffs to PRs when it contains secrets

oh found it

--suppress-secrets

@Andriy Knysh (Cloud Posse) do you have some repos where codefresh is performing helm commands?

@Erik Osterman (Cloud Posse) was working on this one https://github.com/cloudposse/statup/blob/add-helmfile/codefresh/pull-request.yml#L93

the pipeline executes this helmfile https://github.com/cloudposse/statup/blob/add-helmfile/helmfile.yaml

this pipeline is for PR closed https://github.com/cloudposse/statup/blob/add-helmfile/codefresh/pull-request-closed.yml

nice

Morning, can someone assist me in setting up the pipeline in codefresh to push to ECR? I cannot seem to get it the push working. I am getting the following error. aused by Error: Failed to get registry credentials; caused by Error: Registry: '#userPassedEcr' could not be found

(i’m headed to bed, but ping me in the morning if you don’t get it resolved)

good night!

Ok so I did a lot of small changes to the codefresh.yml and what I’ve noticed is some of them where the right ones but the step that fails is when pulling from codefreshes internal docker repo.

[SYSTEM] Error: Failed to calculate image name to push; caused by Error: Failed to pull image r.cfcr.io/nikolavelkovski/nikola-dev-ecs@sha256:73f53ca80ea67f96259
dd4e8ed66e99cde9a8da75c6c49ed1e4c1459985c9a4f; caused by Error: Failed to get registry credentials; caused by Error: Registry: '#userPassedEcr' could not be foun

re-running this step works fine

sometimes I need to re-run it 2 times.

Here’s a screenshot. I had to re-execute the push to ecr step 2 times in order to get it to pass without me doing any change.

@Nikola Velkovski can you share your pipeline? (remove all secrets )

there aren’t any just a sec, so I’ve Alex on support who is investigating

This is the stage that it’s failing with the cryptic error

Are you tagging the image and then push the branch and the tags for example? Maybe it fails to push some tags?

from the error message it fails to pull the branch from the internal repo

can you show the pull step?

sure wait

hmm I’ve a build and a push step

the pull is automatic

upon commit

from GitHub?

yes

ok sec

try to add the clone step https://github.com/cloudposse/statup/blob/add-helmfile/codefresh/pull-request.yml#L10

ok I will

@michal.matyjek, @Daren

we improved https://github.com/cloudposse/github-commenter, let me test all the changes and update README with new examples, and we’ll release (today). Added delete-comment-regex arg, so if provided, the module will try to find and delete all previous comments by the regex before creating a new one.

@Daren re:

Interesting, I looked a helm chart PR to see if I could spot how k8s does their automated comments. They are not posted as review comments, they show up as Issue comments. I dont know the difference

We went through all of that, and found many new details which we did not know before :slightly_smiling_face: (and it’s little bit convoluted).

Summary FYI:

GitHub API allows these types of comments:

1. Comments on repos/commits https://developer.github.com/v3/repos/comments
2. Coments on PR reviews https://developer.github.com/v3/pulls/reviews/#create-a-pull-request-review
3. Comments on PR file (at specified position) https://developer.github.com/v3/pulls/comments
4. Comments on issues https://developer.github.com/v3/issues/comments
5. Comments on PRs (in the global section, not on individual files) https://developer.github.com/v3/issues/comments

Since GitHub considers PRs as issues, #4 and #5 are basically the same (use the same API).

The new module release will support all five (Comment type: commit, pr, issue, pr-review or pr-file).

And all five will support the delete-comment-regex arg to delete the previous comments (if provided).

some screenshots FYI to see how all five look in the UI:

comment on commit:

comment on PR review (Comment #3):

Comments on PR file (at specified position) (Comment #5):

comment on issue (Comment #2):

comment on PR (in the global section) (Comment #4):

(all of those comments were created by the new module)

@Andriy Knysh (Cloud Posse) it was solved, it turned out that if I remove the part where I explicitly state the push type and credentials and just use registry: nikola-dev-ecs it works

¯_(ツ)_/¯

nice

so yea, all those credentials should be settings in the ecr registry, not in pipeline steps

well if one has let’s say 50 ecr registries then it gets interesting

ok and another solution which I prefer was to quote the provider param

provider: ecr == provider: ‘ecr’

@Andriy Knysh (Cloud Posse) thanks for the help!

@mallen has joined the channel

@Daren @michal.matyjek we just released https://github.com/cloudposse/github-commenter that supports delete-comment-regex argument

examples of this are here https://github.com/cloudposse/github-commenter#delete-comment-regex-example-1

Thanks!

(and all those args support ENV equivalents)

yep, here https://github.com/cloudposse/github-commenter#usage

@dustinvb do you have a link handy for your claire image scanning?

https://github.com/codefresh-contrib/cfstep-paclair

@dustinvb is there some way to combine this with https://codefresh.io/codefresh-news/test-reporting/

so we can bypass S3 dependency

https://github.com/dustinvanbuskirk/example-voting-app/blob/master/worker/codefresh.yml#L19-L34

Simple step to add.

This is noted in the new issue I filed on the repository.

@richwine has joined the channel

https://github.com/cloudposse/slack-notifier

we received a contribution which adds a SLACK_CHANNEL parameter so the same webhook can be reused

Cool. Check out my bot? It handled gitops demos during KubeCon.

https://github.com/dustinvanbuskirk/cf-demo-bot

this is great!

I want to use this pattern to open PRs against cloudposse/packages to update packages on a cron using scheduled pipelines in codefresh

so much I want to do…. ugh

I hear you, I have a todo to add GitHub releases into the bot and want to move from one bot to two bots and make the commit items a bit more flexible maybe format them in some way.

New CLI Feature: https://codefresh-io.github.io/cli/validation/validate-pipeline-yaml/

@Igor Rodionov

@Igor Rodionov has joined the channel

nice, but I can not now imaging where to use this

We should have this as a step in your pipeline as code stuff

This will validate the rendered templates of the Codefresh yaml are valid

Team is working on a --local option for using local machine for runner for Codefresh builds as well. I’ll post that here as soon as available that will probably be more widely used.

oh that’s sweet

validate - finally….

thanks!

@michal.matyjek you should see the pipeline creation for Codefresh @Igor Rodionov wrote

Cicd creation of Codefresh repos wired up to pipelines

So you don’t need to login to Codefresh to even setup a new service

part of me says “that’s awesome” but part is like “nooooo….”

haha, how come?

i love the idea that setting up a new repo is all in code

versioning I guess

in fact, i even like the idea of creating the repos that way too

yeah

so that they are setup with branch protections

that makes sense

but to me feels like something that should be out-of-the-box from Codefresh, not separate

ya…. kind’a agree

even basic stuff - like autocreating pipelines by the nature of having file named codefresh.yml in the root of the repo….

which I guess the above is also doing

(or a codefresh/ directory of pipelines)

that would be .codefresh/ - this is what we standardized on at Gladly, so everyone please use that

on why “noooo….” - is we’re having discussions internally on how much to abstract from the dev team - and there are some good arguments to not to abstract (would apply to pipeline creation, but also monochart in helm etc.)

there are also good arguments to abstract - for patterns, ability to easily inject stuff etc.

yea, i see what you mean.. getting farther away from understanding how it works

A cli tool so you can run pipelines locally is awesome. Wercker (https://github.com/wercker/wercker) had a cli for local which was nice to use before they got bought

New option --yaml to run Codefresh pipeline using the YAML file from your local machine.

https://codefresh-io.github.io/cli/pipelines/run-pipeline/

New option --local to run Codefresh pipeline using local Docker daemon now available see link above for option on pipeline run from CLI.

wow, kick ass! moving fast……

@Igor Rodionov what do you think about the local option?

@dustinvb does it support shared configurations?

Hello

I haven’t had a chance to work with it yet and check out the shared configurations or other options. I believe right now the only thing not supported is localized volume caching.

That is due out next week I believe.

how it works with shared configurations?

and let’s say - k8s integration

Ah like getting the context?

--local = local development

yea

oh, access to k8s integrations

I’ll try it real fast, I’ve been a bit limited on time.

(not urgent on our end)

You have sparked my interest Sir!

It all appears to work fine

I’ll post my command and output here.

So appears to work flawless with Kube Context etc.

YAML for this pipeline is located here: https://github.com/dustinvanbuskirk/example-voting-app/blob/master/.codefresh/codefresh-dvts.yml

I’m very excited to test this

yea, this is super dope

@fdrescher talk to @Valdemir if you want a demo

@fdrescher has joined the channel

@fdrescher I d be more than happy to help. When would be most convenient for us to meet?

@dustinvb one thing that would be great to have is being able to pause execution - for pipeline debugging purposes - add a parameter that takes step name and execution stops at this step, so that you can connect to the container and investigate things in the given state

yea, like a break point?

ohhhh yea, that is the BEST thing with circle

love being able to exec into a container

(dustin is on vacay)

@michal.matyjek they said you can open feature requests here: https://github.com/codefresh-io/cli/issues

1 for adding that

https://github.com/codefresh-io/cli/issues/254

https://github.com/codefresh-io/cli/issues/255