#codefresh (2019-01)

codefresh

Archive: https://archive.sweetops.com/codefresh/

2019-01-31

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
mkj28/smashing-codefresh

smashing dashboard for Codefresh builds. Contribute to mkj28/smashing-codefresh development by creating an account on GitHub.

michal.matyjek avatar
michal.matyjek

anyone wanting to improve this - forks are more than welcome

mkj28/smashing-codefresh

smashing dashboard for Codefresh builds. Contribute to mkj28/smashing-codefresh development by creating an account on GitHub.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Nice for office wallboards!

Oleg Sucharevich avatar
Oleg Sucharevich

Cool!

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

this is by @michal.matyjek

dustinvb avatar
dustinvb
504s and Latency on Builds and Pipeline pages in UI

Codefresh’s Status Page - 504s and Latency on Builds and Pipeline pages in UI.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

thanks for the heads up

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

thankfully at this very moment, not building.

dustinvb avatar
dustinvb

Wonder if we can just integrate the status page into the channel

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

done

dustinvb avatar
dustinvb

Incident reported earlier has been resolved.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Wierd, we didn’t get the all lear message from the atom feed?

Codefresh Status - Incident History avatar
Codefresh Status - Incident History
08:35:37 PM

504s and Latency on Builds and Pipeline pages in UI Jan 31, 20:14 UTC Resolved - The issue has been identified and resolved. It was due to a high load on the system. We scaled it up and took the corresponding actions to prevent further issues. Jan 31, 20:14 UTC Monitoring - We’ve resolved the issue and are actively monitoring the system. Jan 31, 19:02 UTC Investigating - Our Team is investigating reports with 504 Gateway errors and latency in Codefresh pages.

504s and Latency on Builds and Pipeline pages in UI

Codefresh’s Status Page - 504s and Latency on Builds and Pipeline pages in UI.

dustinvb avatar
dustinvb

That took a bit…

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Yea, guess it doesn’t poll too often

Codefresh Status - Incident History avatar
Codefresh Status - Incident History
11:35:32 PM

GitHub Incident Jan 31, 22:11 UTC Resolved - GitHub has reported this incident as Resolved. More information here: https://www.githubstatus.com/ Jan 31, 22:10 UTC Update - We are continuing to work on a fix for this issue. Jan 31, 22:01 UTC Identified - GitHub has reported an ongoing incident. This could affect your builds. More information here: https://www.githubstatus.com/

GitHub Incident

Codefresh’s Status Page - GitHub Incident.

GitHub Status

Welcome to GitHub’s home for real-time and historical data on system performance.

2019-01-30

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@Igor Rodionov @Maxim Mironenko (Cloud Posse) https://github.com/codefresh-contrib/cfstep-pipeline-creator

Maxim Mironenko (Cloud Posse) avatar
Maxim Mironenko (Cloud Posse)
10:24:09 PM

@Maxim Mironenko (Cloud Posse) has joined the channel

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
cloudposse/codefresh

Codefresh repos and pipelines in code. Contribute to cloudposse/codefresh development by creating an account on GitHub.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

We’ve updated our example-app with our pipelines for unlimited staging, automatic destroy, release promotion, and multiple cluster deploys using tags

1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
cloudposse/example-app

Example application for CI/CD demonstrations of Codefresh - cloudposse/example-app

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

This also has an example of blue/green deploy’s using Istio

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

we use helmfile with helm

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

we use variant as a “deploy cli”

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@daveyu

1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

what’s notable about these updated pipelines is we avoid the classic “main clone” in every pipeline other than build

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

this makes it faster

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

the build artifact always contains the helmfile to deploy. this is an evolution of our statup demo (which is now out of date)

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@dustinvb I want to add now all the clair scanning and other kinds of tests

dustinvb avatar
dustinvb

https://steps.codefresh.io/ should have the most up-to-date documentation some testing steps requires the user to work with vendor to install server side.

Codefresh: FreshSteps

One small step for your YAML, one giant leap for your pipeline.

2019-01-29

mgrube avatar
mgrube

Anyone know how to nest an array in cfstep-helm value override? For reference I am trying to set the extraHostPathMounts for the kiam helm chart https://github.com/helm/charts/blob/master/stable/kiam/values.yaml#L182

helm/charts

Curated applications for Kubernetes. Contribute to helm/charts development by creating an account on GitHub.

mgrube avatar
mgrube

I wound up creating a new repo with just a codefresh.yaml and a values file

helm/charts

Curated applications for Kubernetes. Contribute to helm/charts development by creating an account on GitHub.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

have you looked into helmfile?

mgrube avatar
mgrube

I hadn’t ever heard of it. Looking into it now

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
cloudposse/helmfiles

Comprehensive Distribution of Helmfiles. Works with helmfile.d - cloudposse/helmfiles

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

So we use this both to distribute backing services as well as applications

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

and by “applications” i mean those delivered via a codefresh pipeline (e.g. web apps)

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

here’s our “example app”

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
cloudposse/example-app

Example application for CI/CD demonstrations of Codefresh - cloudposse/example-app

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

the deploy/ folder contains the helm releases

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

We use this exclusively with helm

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

helmfile takes the pain out of dealing with values.yaml and helm releases.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

plus makes it easier to make helm more 12-factor compliant

2019-01-28

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

This is awesome! https://steps.codefresh.io/

Codefresh: FreshSteps

One small step for your YAML, one giant leap for your pipeline.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

It would be cool if there was a way to define like a step.yaml in a repo with sufficient annotations that codefresh could just populate this automatically.

michal.matyjek avatar
michal.matyjek

I second that ^^^

2019-01-25

Daren avatar
Daren

Is it possible to trigger a pipeline via a webhook, and have the pipeline product an artifact that the caller has access to, say a json file

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

ohhhh some artifact scoped to the CF_API_KEY?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

I don’t know the answer. If they did, it would probably related to this new feature: https://codefresh.io/docs/docs/testing/test-reports/

Test reports

How to create and view Test reports in Codefresh

Daren avatar
Daren

@francisco said you can create a named context, then used a (currently) undocumented api call to obtain it. A cronjob pipeline can be used to clean them up

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

is the API call related to the test reports?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

ohhhh i think i understand now

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

you have a pipeline that writes an artifact to the shared volume

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

and you want to know if you could read that artifact from that volume using the codefresh cli?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

(and apparently this is possible using that undocumented call)

dustinvb avatar
dustinvb
codefresh-io/eks-installer

Tool to setup a new EKS cluster and connect to Codefresh - codefresh-io/eks-installer

dustinvb avatar
dustinvb
codefresh-io/eks-installer

Tool to setup a new EKS cluster and connect to Codefresh - codefresh-io/eks-installer

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Oh this is cool

dustinvb avatar
dustinvb

Example API call to get this context from another system: https://github.com/codefresh-io/cli/blob/master/lib/logic/api/context.js#L50-L60

codefresh-io/cli

Codefresh CLI. Contribute to codefresh-io/cli development by creating an account on GitHub.

dustinvb avatar
dustinvb

Since you get the build id when you trigger a pipeline, I’d suggest setting that in the file name and then you’ll be able to have a unique per build file.

2019-01-24

mgrube avatar
mgrube

I am trying to deploy an application, with the image being pulled from cfr. I added a Secret to the helm chart so that the chart would deploy everything needed for the application, but it doesn’t create the secret first. What is the best practice for uploading the secret?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

good question.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

so there are a few ways to handle this.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

If you’re not going to be using a lot of namespaces, it would be most secure to deploy the pull secret manually as a “one off”

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

If you’re using kops, you can provision a cluster-wide pull secret

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

If you’re using helmfile, then you can deploy it by adding a release of this: https://github.com/cloudposse/charts/tree/master/incubator/dockercfg

cloudposse/charts

The “Cloud Posse” Distribution of Kubernetes Applications - cloudposse/charts

dustinvb avatar
dustinvb
Manage your Kubernetes cluster in Codefresh

Codefresh is a Docker-native CI/CD platform. Instantly build , test and deploy Docker images.

dustinvb avatar
dustinvb

These work with any registries you have connected to Codefresh.

dustinvb avatar
dustinvb

Maybe too many options?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

haha

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@dustinvb would be nice if there was a build-step or something we could use to automatically provision the pull secret for a namespace

dustinvb avatar
dustinvb

You can use the CLI as a build step.

mgrube avatar
mgrube

lol, this is all good stuff. So if I am going to have a namespace per dev team or developer, would it be best to have a helm chart that configures the namespace with all secrets needed?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

the downside with provisioning secrets with helm is that they are in clear text if you run helm get values

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

and are also surfaced in clear text in the helm UI in codefresh

mgrube avatar
mgrube

thats good to know

dustinvb avatar
dustinvb

Bah I am all off on my explanation… I mis read what you’re trying to do.

dustinvb avatar
dustinvb

Use helm-secrets plugin

dustinvb avatar
dustinvb

That’s what I used

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

the helm-secrets plugin doesn’t fix it

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

the values are still passed in clear-text to helm

dustinvb avatar
dustinvb

Yes they are indeed but on the Kubernetes side it will create the secret for you using the Helm Chart.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

hrmm

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

or do you mean helm-secrets will create the Secrets resource out-of-band with helm, so that it won’t be visible by helm?

dustinvb avatar
dustinvb

If you don’t want these secrets showing up in a pipeline it is best to circumvent the secret creation in CI/CD something else like https://github.com/Boostport/kubernetes-vault

Boostport/kubernetes-vault

Use Vault to store secrets for Kubernetes! Contribute to Boostport/kubernetes-vault development by creating an account on GitHub.

dustinvb avatar
dustinvb

It is still visible by Helm Releases and the command in the pipeline.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
mumoshu/aws-secret-operator

A Kubernetes operator that automatically creates and updates Kubernetes secrets according to what are stored in AWS Secrets Manager. - mumoshu/aws-secret-operator

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@mumoshu wrote this! I can’t wait to use it. no need to deploy vault.

dustinvb avatar
dustinvb

Nice!

dustinvb avatar
dustinvb

Yeah for AWS this looks great. I don’t have enough access in AWS to create an example of this. If you ever have a pipeline with a helm chart using this I’d like to check it out.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

True… AWS solution.

dustinvb avatar
dustinvb

Mark, do any of these options look viable for your use case?

mgrube avatar
mgrube

I think for now just to get unblocked i might create manually. https://github.com/mumoshu/aws-secret-operator looks pretty cool in the long term though

:--1:3
mgrube avatar
mgrube

Thanks you guys!

michal.matyjek avatar
michal.matyjek

Anyone here used codefresh api to start pipelines? The swagger at codefresh.io/api is not very clear…. or FWIW - anyone integrated Spinnaker with Codefresh?

michal.matyjek avatar
michal.matyjek

cc @Purva

dustinvb avatar
dustinvb

Our CLI is built off of the API. https://github.com/codefresh-io/cli/blob/master/lib/logic/api/pipeline.js#L116-L165

I’ll get your feedback to the team about the API.

codefresh-io/cli

Codefresh CLI. Contribute to codefresh-io/cli development by creating an account on GitHub.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Could be useful if any codefresh customers are using terraform

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@Dan Garfield

Dan Garfield avatar
Dan Garfield
08:11:11 PM

@Dan Garfield has joined the channel

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
Whats New In Codefresh?

A changelog of new features

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

that’s some pretty sweeet feature velocity

2019-01-23

daveyu avatar
daveyu

I’m trying to deploy to a new k8s cluster, and thinking I missed a step somewhere when provisioning it. any clues here? yaml:

title: Deploy with helmfile
stage: Deploy
image: 'cloudposse/build-harness:${{BUILD_HARNESS_VERSION}}'
working_directory: ./
environment:
  - 'IMAGE_TAG=${{SEMVERSION_COMMIT_SHORT}}'
  - 'RELEASE_NAME=${{NAMESPACE}}'
  - APP_API_ROOT=https://${{BACKEND_APP_HOST}}'
  - 'AWS_ACCESS_KEY_ID=${{CHAMBER_ACCESS_KEY_ID}}'
  - 'AWS_SECRET_ACCESS_KEY=${{CHAMBER_SECRET_ACCESS_KEY}}'
commands:
  - make helm/toolbox/upsert
  - chamber exec app -- helmfile sync

log output:

Pulling image cloudposse/build-harness:0.15.3                                                                  
Digest: sha256:48379abbffe7707ce8185839c24ad20830e45fec7092edbf4febf6b2e859df98                                
Status: Image is up to date for cloudposse/build-harness:0.15.3                                                
ERROR: kubectl not installed!                                                                                  
make: *** [//build-harness/modules/helm/Makefile:24: helm/toolbox/upsert] Error 1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

that looksl ike a really old version of the build-harness

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

newer versions ship kubectl

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

oh snap

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

that’s the latest

1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

I can take a look in a few hours

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

about to give a demo of this https://github.com/cloudposse/example-app

cloudposse/example-app

Example application for CI/CD demonstrations of Codefresh - cloudposse/example-app

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Blue/Green deployments with Codefresh and Istio and Helm

daveyu avatar
daveyu

cool. i’ll poke around.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

it uses our “cartrige” style deployment methodology where everything is bundled in the deployment artifact (aka docker image)

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

to get unblocked, you should be able to do:

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

make packages/install/kubectl

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

as a build step

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

@daveyu i think it should be a geodesic image

image: cloudposse/geodesic:${{GEODESIC_VERSION}}
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Though if he does that, then he needs to install the build-harness

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

so it’s neither here nor there

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

split into two steps

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

yea, could possibly do that

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
deploy_helmfile:
    title: Deploy with helmfile
    stage: "Deploy"
    image: cloudposse/geodesic:${{GEODESIC_VERSION}}
    environment:
    - AWS_REGION=us-west-2
    commands:
    # Install or upgrade tiller
    - "make init"
    - "make helm/toolbox/upsert"
    # Deploy chart to cluster using helmfile
    - "chamber exec kops -- helmfile --file config/helmfile.yaml --selector component=app sync --concurrency 1 --args '--wait --timeout=600 --force --reset-values'"
    # Add kiam annotation
    - kubectl annotate namespace ${NAMESPACE} "[iam.amazonaws.com/permitted=.*](http://iam\.amazonaws\.com/permitted=\.\*)" --overwrite
daveyu avatar
daveyu

thanks! i’ll try those options

daveyu avatar
daveyu

fyi this was the example i followed, and it did work before with the same build-harness version. https://github.com/cloudposse/statup/blob/add-helmfile/codefresh/pull-request.yml#L82

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

That’s really odd. I can’t explain it.

2019-01-20

Igor Rodionov avatar
Igor Rodionov

Codefresh in on thoughtworks’ tech radar

Igor Rodionov avatar
Igor Rodionov
Tools | Technology Radar | Thoughtworks attachment image

The Technology Radar quadrant that explores the latest tech trends in the tools software developers use to create, debug, maintain, and support applications

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

nice!

2019-01-18

dustinvb avatar
dustinvb
Approval

How to pause pipelines for manual approval

party_parrot2
2
:1000:2
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

This is awesome!

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
05:53:14 PM
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
05:53:26 PM
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

not sure if you guys have investigated how Travis does their remote exec

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

but it’s via tmate

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

I’ve reproduced the functionality with this code snippet

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
10:27:29 PM
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

this assumes tmate binary is in your path

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

if you run this snippet, you will get this output

dustinvb avatar
dustinvb

I haven’t personally nor heard of anything about this.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
ssh [[email protected]](mailto:[email protected]\.tmate\.io)
ssh [[email protected]](mailto:ro\[email protected]\.tmate\.io)
<https://tmate.io/t/z0Kgk9o3hU8e6pRYE4fU008mc>
<https://tmate.io/t/ro-XGFBNsGrPrVg4vKdPS5xrjW4k>

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

then I can exec into the container

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

so how does this relate to codefresh?

dustinvb avatar
dustinvb

You should tell me.

dustinvb avatar
dustinvb

LOL

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

so if a container step you add sleep inf, that step won’t finish

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

if the step before that runs the script I shared, then it starts a reverse tunnel out to [tmate.io](http://tmate\.io)

dustinvb avatar
dustinvb
Access and debug the pipeline volume image

Codefresh is a Docker-native CI/CD platform. Instantly build , test and deploy Docker images.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

then I can effectively debug my codefresh container running on codefresh

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

ok, this is cool

dustinvb avatar
dustinvb

We need to think about this as another option and see about a new Fresh Step

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

that’s cool!

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

i didn’t know about the composition terminal

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

though that won’t apply to pipelines, right?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

general pipelines

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

so travis has a button that basically says, on next run, start a tmate session

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

i think in the codefresh world, it would almost be like setting a break point

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

only that the container is left running

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@michal.matyjek @Daren what do you think about something like this?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

not sure if you guys have investigated how Travis does their remote exec

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

provided a hack above

michal.matyjek avatar
michal.matyjek

would be awesome to have this built-in otherwise everytime we want to debug we need to modify a step (or inject it into all steps?)

michal.matyjek avatar
michal.matyjek

wtf is my profile…

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

yea, built-in best

michal.matyjek avatar
michal.matyjek

(or make running locally easy - like a button in the Codefresh UI to run the piepeline/step locally with breakpoint so that you can exec into the container)

dustinvb avatar
dustinvb

I think the break point feature in the UI would be great to read it from yaml or be able to click a step in the UI to set a break point during execution.

A FreshStep though might should be a faster to market MVP.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

HOLY MOLY! I have brilliant idea

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

I’m going to add tmate/session as a target in our build-harness

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

then just add make tmate/session in any pipeline step for debugging

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

and it will output a temporary session

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

in the build logs

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

it will block indefinitely until the tmate session exits or a preconfigurable timeout

1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@daveyu

dustinvb avatar
dustinvb

I’ve invited Francisco here from my team. I think this debugging utility talk will interest him.

dustinvb avatar
dustinvb

We just ran into something today this would have aided in. I’d like to get this integrated into the pipeline like we do with cf_export.

daveyu avatar
daveyu

This kind of feature was really useful when I used CircleCi

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
12:31:47 AM

2019-01-17

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

we’ve released our helmfile to install the codefresh runtime environment using our monochart

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
cloudposse/helmfiles

Comprehensive Distribution of Helmfiles. Works with helmfile.d - cloudposse/helmfiles

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@dustinvb

cloudposse/helmfiles

Comprehensive Distribution of Helmfiles. Works with helmfile.d - cloudposse/helmfiles

mumoshu avatar
mumoshu
08:39:10 AM

@mumoshu has joined the channel

dustinvb avatar
dustinvb

We also have venona now which does behind the firewall runtimes with no ingress.

https://github.com/codefresh-io/venona

codefresh-io/venona

Codefresh runtime-environment agent. Contribute to codefresh-io/venona development by creating an account on GitHub.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Cool, we will do the same for that

dustinvb avatar
dustinvb

The manual approval, step is due out next week. You’ll already see in Slack there is an option to send notifications about approval builds and a new option on the build screen to the right of pending for builds waiting approval.

2
2
:1000:2
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

That will be awesome!!

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

So an engineer can kick off the deployment by cutting a release in github

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

that triggers a codefresh pipeline

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

which requests an approval via slack to the engineering manager

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

if he approves it, then the deployment proceeds.

dustinvb avatar
dustinvb

Yes but permissions are not available yet around the process but the pipeline is stamped with the user who approved/denied.

dustinvb avatar
dustinvb

@mgrube Join us here.

:--1:1
dustinvb avatar
dustinvb

Ah I see you, didn’t say you joined.

mgrube avatar
mgrube

Clearly, I am a ninja

dustinvb avatar
dustinvb

With that professional incognito profile pic I would say that is 100% the case.

1

2019-01-10

dustinvb avatar
dustinvb

Bah that link requires login LOL.

dustinvb avatar
dustinvb

Have you seen the new metrics tab on the bottom of your builds we pushed yesterday?

The main one shows cumulative and you can also see each step there or click on the step and get more details.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

The stats are just so freaking awesome!!

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

I did see them the other day

dustinvb avatar
dustinvb
04:50:51 PM
dustinvb avatar
dustinvb
04:51:06 PM
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

That link is just medium being medium. There’s a X in the upper right corner that closes at dialog. Stupid medium.

dustinvb avatar
dustinvb

Also our first version of our behind the firewall agent has been released. https://github.com/codefresh-io/venona You can deploy to your Kubernetes cluster and use as a custom runtime.

codefresh-io/venona

Codefresh runtime-environment agent. Contribute to codefresh-io/venona development by creating an account on GitHub.

party_parrot1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Dustin how is that different from using the codefresh cli deploy the run time environment?

dustinvb avatar
dustinvb

That requires you allow ingress from Codefresh SaaS

dustinvb avatar
dustinvb

This would be where you don’t want to expose your Kubernetes API to internet.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

That’s badass

dustinvb avatar
dustinvb

Sorry API proxy is coming soon. This allows for the runtime to be behind the firewall in our Hybrid scenario.

2019-01-09

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@dustinvb have you seen #kubecost? I think this could add some extra bling to your demos.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

They have a dashboard that shows exactly how much a deployment costs plus makes it easy to “right size” pods.

2019-01-08

joshmyers avatar
joshmyers

make is the new hotness, until you get too deep into it. Is it a really cool project these days if it isn’t go & make ?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

haha

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

yea, stay on the surface

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

the deep waters of make are dangerous

joshmyers avatar
joshmyers

Likely not useful, but wrote https://github.com/joshmyers/dynolocker for a quick distributed lock before

joshmyers/dynolocker

A CLI tool for distributed locks using DynamoDB. Contribute to joshmyers/dynolocker development by creating an account on GitHub.

:--1:1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

oh that’s cool!

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

yea, that’s basically what I wanted

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

you sir got your first

1

2019-01-07

michal.matyjek avatar
michal.matyjek

Wondering if anyone solved/hit these:

1) discarding concurrent builds - imagine I want to have pipeline triggered on git commit, it takes 1 hour to run, but if any other build of the pipeline is triggered we only keep the currently building one and the most recent one, all others are discarded.

2) pausing pipeline - if we have pipeline triggered on commit or on cron - is there a way to manually pause the schedule? (other than deleting the trigger? since deleting/re-creating trigger is involved). Looking for something similar to Jenkins enable/disable pipeline button

3) what do people use for sequential/unique build numbering - if we wanted to generate semantic versions for our builds?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

i’ve wanted to have “locks”

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

so we can have a pipeline block untili it obtains the lock.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

i’ve seen semver services

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

e.g. you can curl an API for a given service and specify if you want a new path, minor or major release

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

e.g. curl <https://semver.host.io?service=foobar&patch+=1>

michal.matyjek avatar
michal.matyjek

interesting… not exactly what I wanted but makes sense

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

one of our problems is with helm versioning

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

basically different projects have pinned to different versions of our release harness

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

which mean different versions of helm

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

one option is to always check the server and then use that version locally

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
12:43:37 AM

on an totally unrelated note, I was surprised to see how many people we’ve found who like make

:--1:1

2019-01-03

dustinvb avatar
dustinvb

CLI can now use your local machine as the volume for more verbose debugging.

--local-volume	--lv		Use your file system as volume in local run
:--1:4
fiesta_parrot1
    keyboard_arrow_up