#codefresh (2019-01)
Archive: https://archive.sweetops.com/codefresh/
2019-01-03

CLI can now use your local machine as the volume for more verbose debugging.
--local-volume --lv Use your file system as volume in local run

2019-01-07

Wondering if anyone solved/hit these:
1) discarding concurrent builds - imagine I want to have pipeline triggered on git commit, it takes 1 hour to run, but if any other build of the pipeline is triggered we only keep the currently building one and the most recent one, all others are discarded.
2) pausing pipeline - if we have pipeline triggered on commit or on cron - is there a way to manually pause the schedule? (other than deleting the trigger? since deleting/re-creating trigger is involved). Looking for something similar to Jenkins enable/disable pipeline button
3) what do people use for sequential/unique build numbering - if we wanted to generate semantic versions for our builds?

i’ve wanted to have “locks”

so we can have a pipeline block untili it obtains the lock.

i’ve seen semver services

e.g. you can curl an API for a given service and specify if you want a new path, minor or major release

e.g. curl <https://semver.host.io?service=foobar&patch+=1>

interesting… not exactly what I wanted but makes sense

one of our problems is with helm versioning

basically different projects have pinned to different versions of our release harness

which mean different versions of helm

one option is to always check the server and then use that version locally

on an totally unrelated note, I was surprised to see how many people we’ve found who like make
2019-01-08

make is the new hotness, until you get too deep into it. Is it a really cool project these days if it isn’t go & make ?

haha

yea, stay on the surface

the deep waters of make are dangerous

Likely not useful, but wrote https://github.com/joshmyers/dynolocker for a quick distributed lock before
A CLI tool for distributed locks using DynamoDB. Contribute to joshmyers/dynolocker development by creating an account on GitHub.

oh that’s cool!

yea, that’s basically what I wanted

2019-01-09

@dustinvb have you seen #kubecost? I think this could add some extra bling to your demos.

They have a dashboard that shows exactly how much a deployment costs plus makes it easy to “right size” pods.
2019-01-10

Bah that link requires login LOL.

Have you seen the new metrics tab on the bottom of your builds we pushed yesterday?
The main one shows cumulative and you can also see each step there or click on the step and get more details.

The stats are just so freaking awesome!!

I did see them the other day



That link is just medium being medium. There’s a X in the upper right corner that closes at dialog. Stupid medium.

Also our first version of our behind the firewall agent has been released. https://github.com/codefresh-io/venona You can deploy to your Kubernetes cluster and use as a custom runtime.
Codefresh runtime-environment agent. Contribute to codefresh-io/venona development by creating an account on GitHub.


Dustin how is that different from using the codefresh cli deploy the run time environment?

That requires you allow ingress from Codefresh SaaS

This would be where you don’t want to expose your Kubernetes API to internet.

That’s badass

Sorry API proxy is coming soon. This allows for the runtime to be behind the firewall in our Hybrid scenario.
2019-01-17

–

we’ve released our helmfile to install the codefresh
runtime environment using our monochart

Comprehensive Distribution of Helmfiles. Works with helmfile.d
- cloudposse/helmfiles

@dustinvb
Comprehensive Distribution of Helmfiles. Works with helmfile.d
- cloudposse/helmfiles

@mumoshu has joined the channel

We also have venona now which does behind the firewall runtimes with no ingress.
Codefresh runtime-environment agent. Contribute to codefresh-io/venona development by creating an account on GitHub.

Cool, we will do the same for that

The manual approval, step is due out next week. You’ll already see in Slack there is an option to send notifications about approval builds and a new option on the build screen to the right of pending for builds waiting approval.


That will be awesome!!

So an engineer can kick off the deployment by cutting a release in github

that triggers a codefresh pipeline

which requests an approval via slack to the engineering manager

if he approves it, then the deployment proceeds.

Yes but permissions are not available yet around the process but the pipeline is stamped with the user who approved/denied.


Ah I see you, didn’t say you joined.

Clearly, I am a ninja

2019-01-18


This is awesome!



not sure if you guys have investigated how Travis does their remote exec

but it’s via tmate

I’ve reproduced the functionality with this code snippet


this assumes tmate
binary is in your path

if you run this snippet, you will get this output

I haven’t personally nor heard of anything about this.

ssh [email protected]
ssh [email protected]
<https://tmate.io/t/z0Kgk9o3hU8e6pRYE4fU008mc>
<https://tmate.io/t/ro-XGFBNsGrPrVg4vKdPS5xrjW4k>

then I can exec into the container

so how does this relate to codefresh?

You should tell me.

LOL

so if a container step you add sleep inf
, that step won’t finish

if the step before that runs the script I shared, then it starts a reverse tunnel out to [tmate.io](http://tmate.io)

Codefresh is a Docker-native CI/CD platform. Instantly build , test and deploy Docker images.

then I can effectively debug my codefresh container running on codefresh

ok, this is cool

We need to think about this as another option and see about a new Fresh Step

that’s cool!

i didn’t know about the composition terminal

though that won’t apply to pipelines, right?

general pipelines

so travis has a button that basically says, on next run, start a tmate session

i think in the codefresh world, it would almost be like setting a break point

only that the container is left running

@michal.matyjek @Daren what do you think about something like this?

not sure if you guys have investigated how Travis does their remote exec

provided a hack above

would be awesome to have this built-in otherwise everytime we want to debug we need to modify a step (or inject it into all steps?)

wtf is my profile…

yea, built-in best

(or make running locally easy - like a button in the Codefresh UI to run the piepeline/step locally with breakpoint so that you can exec into the container)

I think the break point feature in the UI would be great to read it from yaml or be able to click a step in the UI to set a break point during execution.
A FreshStep though might should be a faster to market MVP.

HOLY MOLY! I have brilliant idea

I’m going to add tmate/session
as a target in our build-harness

then just add make tmate/session
in any pipeline step for debugging

and it will output a temporary session

in the build logs

it will block indefinitely until the tmate
session exits or a preconfigurable timeout

@daveyu

I’ve invited Francisco here from my team. I think this debugging utility talk will interest him.

We just ran into something today this would have aided in. I’d like to get this integrated into the pipeline like we do with cf_export.

This kind of feature was really useful when I used CircleCi

2019-01-20

Codefresh in on thoughtworks’ tech radar


The Technology Radar quadrant that explores the latest tech trends in the tools software developers use to create, debug, maintain, and support applications

nice!
2019-01-23

I’m trying to deploy to a new k8s cluster, and thinking I missed a step somewhere when provisioning it. any clues here? yaml:
title: Deploy with helmfile
stage: Deploy
image: 'cloudposse/build-harness:${{BUILD_HARNESS_VERSION}}'
working_directory: ./
environment:
- 'IMAGE_TAG=${{SEMVERSION_COMMIT_SHORT}}'
- 'RELEASE_NAME=${{NAMESPACE}}'
- APP_API_ROOT=https://${{BACKEND_APP_HOST}}'
- 'AWS_ACCESS_KEY_ID=${{CHAMBER_ACCESS_KEY_ID}}'
- 'AWS_SECRET_ACCESS_KEY=${{CHAMBER_SECRET_ACCESS_KEY}}'
commands:
- make helm/toolbox/upsert
- chamber exec app -- helmfile sync
log output:
Pulling image cloudposse/build-harness:0.15.3
Digest: sha256:48379abbffe7707ce8185839c24ad20830e45fec7092edbf4febf6b2e859df98
Status: Image is up to date for cloudposse/build-harness:0.15.3
ERROR: kubectl not installed!
make: *** [//build-harness/modules/helm/Makefile:24: helm/toolbox/upsert] Error 1

that looksl ike a really old version of the build-harness

newer versions ship kubectl

oh snap


I can take a look in a few hours

about to give a demo of this https://github.com/cloudposse/example-app
Example application for CI/CD demonstrations of Codefresh - cloudposse/example-app

Blue/Green deployments with Codefresh and Istio and Helm

cool. i’ll poke around.

it uses our “cartrige” style deployment methodology where everything is bundled in the deployment artifact (aka docker image)

to get unblocked, you should be able to do:

make packages/install/kubectl

as a build step

@daveyu i think it should be a geodesic
image
image: cloudposse/geodesic:${{GEODESIC_VERSION}}

Though if he does that, then he needs to install the build-harness

so it’s neither here nor there

split into two steps

yea, could possibly do that

deploy_helmfile:
title: Deploy with helmfile
stage: "Deploy"
image: cloudposse/geodesic:${{GEODESIC_VERSION}}
environment:
- AWS_REGION=us-west-2
commands:
# Install or upgrade tiller
- "make init"
- "make helm/toolbox/upsert"
# Deploy chart to cluster using helmfile
- "chamber exec kops -- helmfile --file config/helmfile.yaml --selector component=app sync --concurrency 1 --args '--wait --timeout=600 --force --reset-values'"
# Add kiam annotation
- kubectl annotate namespace ${NAMESPACE} "iam.amazonaws.com/permitted=.*" --overwrite

thanks! i’ll try those options

fyi this was the example i followed, and it did work before with the same build-harness version. https://github.com/cloudposse/statup/blob/add-helmfile/codefresh/pull-request.yml#L82

That’s really odd. I can’t explain it.
2019-01-24

I am trying to deploy an application, with the image being pulled from cfr. I added a Secret to the helm chart so that the chart would deploy everything needed for the application, but it doesn’t create the secret first. What is the best practice for uploading the secret?

good question.

so there are a few ways to handle this.

If you’re not going to be using a lot of namespaces, it would be most secure to deploy the pull secret manually as a “one off”

If you’re using kops
, you can provision a cluster-wide pull secret

If you’re using helmfile
, then you can deploy it by adding a release of this: https://github.com/cloudposse/charts/tree/master/incubator/dockercfg
The “Cloud Posse” Distribution of Kubernetes Applications - cloudposse/charts

Mock deploy a secret using our Services page: https://codefresh.io/docs/docs/deploy-to-kubernetes/manage-kubernetes/#deploy-new-service
Use Codefresh CLI: https://codefresh-io.github.io/cli/more/image-pull-secret/
Codefresh is a Docker-native CI/CD platform. Instantly build , test and deploy Docker images.

These work with any registries you have connected to Codefresh.

Maybe too many options?

haha

@dustinvb would be nice if there was a build-step or something we could use to automatically provision the pull secret for a namespace

You can use the CLI as a build step.

lol, this is all good stuff. So if I am going to have a namespace per dev team or developer, would it be best to have a helm chart that configures the namespace with all secrets needed?

the downside with provisioning secrets with helm
is that they are in clear text if you run helm get values

and are also surfaced in clear text in the helm
UI in codefresh

thats good to know

Bah I am all off on my explanation… I mis read what you’re trying to do.

Use helm-secrets plugin

That’s what I used

the helm-secrets
plugin doesn’t fix it

the values are still passed in clear-text to helm

Yes they are indeed but on the Kubernetes side it will create the secret for you using the Helm Chart.

hrmm

or do you mean helm-secrets
will create the Secrets
resource out-of-band with helm
, so that it won’t be visible by helm
?

If you don’t want these secrets showing up in a pipeline it is best to circumvent the secret creation in CI/CD something else like https://github.com/Boostport/kubernetes-vault
Use Vault to store secrets for Kubernetes! Contribute to Boostport/kubernetes-vault development by creating an account on GitHub.

It is still visible by Helm Releases and the command in the pipeline.

@dustinvb check this out: https://github.com/mumoshu/aws-secret-operator
A Kubernetes operator that automatically creates and updates Kubernetes secrets according to what are stored in AWS Secrets Manager. - mumoshu/aws-secret-operator

@mumoshu wrote this! I can’t wait to use it. no need to deploy vault.

Nice!

Yeah for AWS this looks great. I don’t have enough access in AWS to create an example of this. If you ever have a pipeline with a helm chart using this I’d like to check it out.

True… AWS solution.

Mark, do any of these options look viable for your use case?

I think for now just to get unblocked i might create manually. https://github.com/mumoshu/aws-secret-operator looks pretty cool in the long term though

Thanks you guys!

Anyone here used codefresh api to start pipelines? The swagger at codefresh.io/api is not very clear…. or FWIW - anyone integrated Spinnaker with Codefresh?

cc @Purva

Our CLI is built off of the API. https://github.com/codefresh-io/cli/blob/master/lib/logic/api/pipeline.js#L116-L165
I’ll get your feedback to the team about the API.
Codefresh CLI. Contribute to codefresh-io/cli development by creating an account on GitHub.

1 vote and 0 comments so far on Reddit

Could be useful if any codefresh customers are using terraform

@Dan Garfield

@Dan Garfield has joined the channel

A changelog of new features

that’s some pretty sweeet feature velocity
2019-01-25

Is it possible to trigger a pipeline via a webhook, and have the pipeline product an artifact that the caller has access to, say a json file

ohhhh some artifact scoped to the CF_API_KEY
?

I don’t know the answer. If they did, it would probably related to this new feature: https://codefresh.io/docs/docs/testing/test-reports/
How to create and view Test reports in Codefresh

@francisco said you can create a named context, then used a (currently) undocumented api call to obtain it. A cronjob pipeline can be used to clean them up

is the API call related to the test reports?

ohhhh i think i understand now

you have a pipeline that writes an artifact to the shared volume

and you want to know if you could read that artifact from that volume using the codefresh cli?

(and apparently this is possible using that undocumented call)

Example of writing a file to context. https://github.com/codefresh-io/eks-installer/blob/master/scripts/codefresh-save-tfstate.sh
Tool to setup a new EKS cluster and connect to Codefresh - codefresh-io/eks-installer

Using that at a later date https://github.com/codefresh-io/eks-installer/blob/master/scripts/codefresh-load-tfstate.sh
Tool to setup a new EKS cluster and connect to Codefresh - codefresh-io/eks-installer

Oh this is cool

Example API call to get this context from another system: https://github.com/codefresh-io/cli/blob/master/lib/logic/api/context.js#L50-L60
Codefresh CLI. Contribute to codefresh-io/cli development by creating an account on GitHub.

Since you get the build id when you trigger a pipeline, I’d suggest setting that in the file name and then you’ll be able to have a unique per build file.
2019-01-28

This is awesome! https://steps.codefresh.io/
One small step for your YAML, one giant leap for your pipeline.

It would be cool if there was a way to define like a step.yaml
in a repo with sufficient annotations that codefresh could just populate this automatically.

I second that ^^^
2019-01-29

Anyone know how to nest an array in cfstep-helm value override? For reference I am trying to set the extraHostPathMounts for the kiam helm chart https://github.com/helm/charts/blob/master/stable/kiam/values.yaml#L182
Curated applications for Kubernetes. Contribute to helm/charts development by creating an account on GitHub.

I wound up creating a new repo with just a codefresh.yaml and a values file
Curated applications for Kubernetes. Contribute to helm/charts development by creating an account on GitHub.

have you looked into helmfile
?

I hadn’t ever heard of it. Looking into it now

Comprehensive Distribution of Helmfiles. Works with helmfile.d
- cloudposse/helmfiles

So we use this both to distribute backing services as well as applications

and by “applications” i mean those delivered via a codefresh pipeline (e.g. web apps)

here’s our “example app”

Example application for CI/CD demonstrations of Codefresh - cloudposse/example-app

the deploy/
folder contains the helm releases

We use this exclusively with helm

helmfile takes the pain out of dealing with values.yaml
and helm releases.

plus makes it easier to make helm
more 12-factor compliant
2019-01-30

@Igor Rodionov @Maxim Mironenko (Cloud Posse) https://github.com/codefresh-contrib/cfstep-pipeline-creator

@Maxim Mironenko (Cloud Posse) has joined the channel

How does this compare to https://github.com/cloudposse/codefresh/tree/initial-implementation
Codefresh repos and pipelines in code. Contribute to cloudposse/codefresh development by creating an account on GitHub.

We’ve updated our example-app
with our pipelines for unlimited staging, automatic destroy, release promotion, and multiple cluster deploys using tags

Example application for CI/CD demonstrations of Codefresh - cloudposse/example-app

This also has an example of blue/green deploy’s using Istio

we use helmfile
with helm

we use variant
as a “deploy cli”


what’s notable about these updated pipelines is we avoid the classic “main clone” in every pipeline other than build

this makes it faster

the build artifact always contains the helmfile
to deploy. this is an evolution of our statup
demo (which is now out of date)

@dustinvb I want to add now all the clair scanning and other kinds of tests

https://steps.codefresh.io/ should have the most up-to-date documentation some testing steps requires the user to work with vendor to install server side.
One small step for your YAML, one giant leap for your pipeline.
2019-01-31

smashing dashboard for Codefresh builds. Contribute to mkj28/smashing-codefresh development by creating an account on GitHub.

anyone wanting to improve this - forks are more than welcome
smashing dashboard for Codefresh builds. Contribute to mkj28/smashing-codefresh development by creating an account on GitHub.

Nice for office wallboards!

Cool!

this is by @michal.matyjek

Codefresh’s Status Page - 504s and Latency on Builds and Pipeline pages in UI.

thanks for the heads up

thankfully at this very moment, not building.

Wonder if we can just integrate the status page into the channel

done

Incident reported earlier has been resolved.

Wierd, we didn’t get the all lear message from the atom feed?

504s and Latency on Builds and Pipeline pages in UI Jan 31, 20:14 UTC Resolved - The issue has been identified and resolved. It was due to a high load on the system. We scaled it up and took the corresponding actions to prevent further issues. Jan 31, 20:14 UTC Monitoring - We’ve resolved the issue and are actively monitoring the system. Jan 31, 19:02 UTC Investigating - Our Team is investigating reports with 504 Gateway errors and latency in Codefresh pages.
Codefresh’s Status Page - 504s and Latency on Builds and Pipeline pages in UI.

That took a bit…

Yea, guess it doesn’t poll too often

GitHub Incident Jan 31, 22:11 UTC Resolved - GitHub has reported this incident as Resolved. More information here: https://www.githubstatus.com/ Jan 31, 22:10 UTC Update - We are continuing to work on a fix for this issue. Jan 31, 22:01 UTC Identified - GitHub has reported an ongoing incident. This could affect your builds. More information here: https://www.githubstatus.com/
Codefresh’s Status Page - GitHub Incident.
Welcome to GitHub’s home for real-time and historical data on system performance.