#codefresh (2019-03)

codefresh

Archive: https://archive.sweetops.com/codefresh/

2019-03-28

oscarsullivan_old avatar
oscarsullivan_old

I’m a bit stuck. I setup my EKS cluster with the CP module. I’ve connected it to Codefresh. I’ve created a Codefresh service to launch my image. I get an error saying no nodes available. I check EC2 console and I have two worker instances (that’s a synonym of node, right).

Does anyone have any insight on what I may have missed from just reading that and seeing the screenshots below? thanks

oscarsullivan_old avatar
oscarsullivan_old
oscarsullivan_old avatar
oscarsullivan_old

EKS cluster

oscarsullivan_old avatar
oscarsullivan_old
Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

if you run kubectl get pods --all-namespaces, what do you see?

oscarsullivan_old avatar
oscarsullivan_old
~ ⨠ kubectl --kubeconfig="/conf/kube/eks" get pods --all-namespaces
error: the server doesn't have a resource type "pods"
oscarsullivan_old avatar
oscarsullivan_old

hmm

oscarsullivan_old avatar
oscarsullivan_old

oh wait sorry my token expired

oscarsullivan_old avatar
oscarsullivan_old
kubectl --kubeconfig="/conf/kube/eks" get pods --all-namespaces
NAMESPACE     NAME                       READY   STATUS    RESTARTS   AGE
default       pod-68b9fd45b4-w88jb       0/1     Pending   0          40m
kube-public   pod-78c67897fc-tvg28       0/1     Pending   0          45m
kube-system   coredns-854797898c-jhkfd   0/1     Pending   0          3h
kube-system   coredns-854797898c-wp4vz   0/1     Pending   0          3h
oscarsullivan_old avatar
oscarsullivan_old

what’s weird is that I deployed pod onto default and kube-public a wihle ago

oscarsullivan_old avatar
oscarsullivan_old

it is since removed form kube-public in codefresh though

oscarsullivan_old avatar
oscarsullivan_old

and kube-dns appears in codefresh portal with replicas 0/2

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@Andriy Knysh (Cloud Posse) others have had trouble with getting nodes registered

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

What does get nodes show?

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

kubectl get nodes

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

(On phone)

oscarsullivan_old avatar
oscarsullivan_old
kubectl --kubeconfig="/conf/kube/eks" get nodes --all-namespaces
No resources found.
Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
cloudposse/terraform-aws-eks-cluster

Terraform module for provisioning an EKS cluster. Contribute to cloudposse/terraform-aws-eks-cluster development by creating an account on GitHub.

oscarsullivan_old avatar
oscarsullivan_old

Hmm no it’s actually a CP project Erik pointed me to last week

oscarsullivan_old avatar
oscarsullivan_old

ah wait

oscarsullivan_old avatar
oscarsullivan_old

yeh that example

oscarsullivan_old avatar
oscarsullivan_old

that’s it

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

for worker nodes to join the cluster, you need to apply that config map

oscarsullivan_old avatar
oscarsullivan_old

Right that adds up with just what I was reading here https://learn.hashicorp.com/terraform/aws/eks-intro#configuring-kubectl-for-eks

AWS EKS Introduction | Terraform - HashiCorp Learn attachment image

A Terraform configuration based introduction to EKS.

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

either using the example code, or manually

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

let me know if that still is not working

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

it was tested about 2 months ago, was ok

oscarsullivan_old avatar
oscarsullivan_old

thanks will follow this:


\###########################################################################################################################################

\#

\# NOTE: To automatically apply the Kubernetes configuration to the cluster (which allows the worker nodes to join the cluster),

\# the requirements outlined here must be met:

\# <https://www.terraform.io/docs/providers/aws/guides/eks-getting-started.html#preparation>

\# <https://www.terraform.io/docs/providers/aws/guides/eks-getting-started.html#configuring-kubectl-for-eks>

\# <https://www.terraform.io/docs/providers/aws/guides/eks-getting-started.html#required-kubernetes-configuration-to-join-worker-nodes>

\#

\# If you want to automatically apply the Kubernetes configuration, set `var.apply_config_map_aws_auth` to "true"

\#

\###########################################################################################################################################
Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

but recently a few people reported issues with workers joining the cluster

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
node not joining cluster · Issue #13 · cloudposse/terraform-aws-eks-cluster

Hi, during the deploymnet everithing went well, but when i tryed to query the cluster i get kubectl get no No resources found. this are the logs from the node. Mar 19 14:01:01 ip-172-18-14-119 kube…

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

so if you have the same issue, I’ll have to look into that

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@Andriy Knysh (Cloud Posse)

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

i saw that

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

as i mentioned, we did not see issues like that before

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

@oscarsullivan_old try the example and let us know

oscarsullivan_old avatar
oscarsullivan_old

oscarsullivan_old avatar
oscarsullivan_old

where do you store the equivalent of ~/.kube/config for geodesic so you dont have to specify it?

oscarsullivan_old avatar
oscarsullivan_old

that would technically be /conf/.kube/config correct?

oscarsullivan_old avatar
oscarsullivan_old


don’t have to
e.g kubectl –kubeconfig=”/conf/kube/eks”get nodes –all-namespaces

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

i think geodesic has an ENV var for that, @Erik Osterman (Cloud Posse) do you remember which one?

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

(or maybe it’s just for kops)

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

It’s a kubectl env

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Yes

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

That one

oscarsullivan_old avatar
oscarsullivan_old

so if I set ENV KUBECONFIG=/conf/kube/eks should be ok?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@Andriy Knysh (Cloud Posse) what’s the path setting g for the module

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

That is what should change to dev shm

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
cloudposse/terraform-aws-eks-cluster

Terraform module for provisioning an EKS cluster. Contribute to cloudposse/terraform-aws-eks-cluster development by creating an account on GitHub.

oscarsullivan_old avatar
oscarsullivan_old

groovy that worked

oscarsullivan_old avatar
oscarsullivan_old


so if I set ENV KUBECONFIG=/conf/kube/eks should be ok?

oscarsullivan_old avatar
oscarsullivan_old

ok let me try the example proj now

oscarsullivan_old avatar
oscarsullivan_old
 ⧉  sandbox (aws:default)
 ✓   (-sandbox-admin) ~ ⨠ kubectl get nodes --all-namespaces
No resources found.
 ⧉  sandbox (aws:default)
 ✓   (-sandbox-admin) ~ ⨠ kubectl get pods --all-namespaces
NAMESPACE     NAME                       READY   STATUS    RESTARTS   AGE
default       pod-68b9fd45b4-w88jb       0/1     Pending   0          54m
kube-public   pod-78c67897fc-tvg28       0/1     Pending   0          59m
kube-system   coredns-854797898c-jhkfd   0/1     Pending   0          3h
kube-system   coredns-854797898c-wp4vz   0/1     Pending   0          3h
Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

with var.apply_config_map_aws_auth` set to “true”

oscarsullivan_old avatar
oscarsullivan_old

ok let me try

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@Andriy Knysh (Cloud Posse) Where does the module write the config?

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

you specify the path

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Can we specify then dev shm?

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

kubeconfig_filename = "${path.module}/kubeconfig${var.delimiter}${module.eks_cluster.eks_cluster_id}.yaml"

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

need to change that to a var

oscarsullivan_old avatar
oscarsullivan_old
 ✓   (healthera-sandbox-admin) eks ⨠ kubectl get nodes --all-namespaces
NAME                                          STATUS     ROLES    AGE   VERSION
ip-10-14-108-66.eu-west-2.compute.internal    NotReady   <none>   10s   v1.12.7
ip-10-14-152-132.eu-west-2.compute.internal   NotReady   <none>   10s   v1.12.7
oscarsullivan_old avatar
oscarsullivan_old

@Andriy Knysh (Cloud Posse) getting closer

oscarsullivan_old avatar
oscarsullivan_old

cool so kube-dns has deployed now

oscarsullivan_old avatar
oscarsullivan_old
Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

do you see the workers?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

See above

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@Andriy Knysh (Cloud Posse) can you update the issue with this new information ?

oscarsullivan_old avatar
oscarsullivan_old
 ✓   (healthera-sandbox-admin) eks ⨠ kubectl get nodes --all-namespaces
NAME                                          STATUS   ROLES    AGE   VERSION
ip-10-14-108-66.eu-west-2.compute.internal    Ready    <none>   47s   v1.12.7
ip-10-14-152-132.eu-west-2.compute.internal   Ready    <none>   47s   v1.12.7
 ⧉  sandbox (aws:default)
 ✓   (-sandbox-admin) eks ⨠ kubectl get nodes --all-namespaces
NAME                                          STATUS   ROLES    AGE   VERSION
ip-10-14-108-66.eu-west-2.compute.internal    Ready    <none>   1m    v1.12.7
ip-10-14-152-132.eu-west-2.compute.internal   Ready    <none>   1m    v1.12.7
 ⧉  sandbox (aws:default)
 ✓   (-sandbox-admin) eks ⨠ kubectl get pods --all-namespaces
NAMESPACE     NAME                       READY   STATUS             RESTARTS   AGE
default       pod-68b9fd45b4-w88jb       0/1     ImagePullBackOff   0          1h
kube-public   pod-78c67897fc-tvg28       0/1     ImagePullBackOff   0          1h
kube-system   aws-node-mzj4z             1/1     Running            0          1m
kube-system   aws-node-vzj9j             1/1     Running            0          1m
kube-system   coredns-854797898c-jhkfd   1/1     Running            0          4h
kube-system   coredns-854797898c-wp4vz   1/1     Running            0          4h
kube-system   kube-proxy-4b7r7           1/1     Running            0          1m
kube-system   kube-proxy-hxkng           1/1     Running            0          1m
Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

i can update the issue, yes

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

but i pointed out a few times there to do what @oscarsullivan_old just did

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

so either that did not work for them, or something else was wrong

oscarsullivan_old avatar
oscarsullivan_old

yeh how come that isn’t true by default

oscarsullivan_old avatar
oscarsullivan_old

who wouldn’t want their workesr auto connecting???

oscarsullivan_old avatar
oscarsullivan_old
apply_config_map_aws_auth="true"

I just added this to my terraform.auto.tfvars

oscarsullivan_old avatar
oscarsullivan_old

ok I’ve got somethingI can try

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

as i remember there were some thoughts around that, but don’t remember exactly the details

oscarsullivan_old avatar
oscarsullivan_old

workinggg

oscarsullivan_old avatar
oscarsullivan_old

codefresh docs a bit dated

oscarsullivan_old avatar
oscarsullivan_old

the ui said just input the [image]:[tag]

oscarsullivan_old avatar
oscarsullivan_old

but hte docs said the full path

oscarsullivan_old avatar
oscarsullivan_old

oh wait no lol still failed

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

is it your image?

oscarsullivan_old avatar
oscarsullivan_old
oscarsullivan_old avatar
oscarsullivan_old

I felt I had correctly linked codefresh, codefresh repo, codefresh pipeline pushing to repo, and EKR to codefresh correctly

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

i think you need to deploy Docker pull secrets to k8s https://github.com/cloudposse/charts/tree/master/incubator/dockercfg

cloudposse/charts

The “Cloud Posse” Distribution of Kubernetes Applications - cloudposse/charts

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

1 sec

oscarsullivan_old avatar
oscarsullivan_old

It doesn’t have any secrets

oscarsullivan_old avatar
oscarsullivan_old

(if that’s a valid answer)

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
cloudposse/example-app

Example application for CI/CD demonstrations of Codefresh - cloudposse/example-app

oscarsullivan_old avatar
oscarsullivan_old

That’s even when it is codefresh’s registry?

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

yes

oscarsullivan_old avatar
oscarsullivan_old

Alright thanks

oscarsullivan_old avatar
oscarsullivan_old

will look at htose last 3 links and report back

oscarsullivan_old avatar
oscarsullivan_old

heh I’m using the GUI

oscarsullivan_old avatar
oscarsullivan_old

ok let’s try that

oscarsullivan_old avatar
oscarsullivan_old

super easy with GUI

oscarsullivan_old avatar
oscarsullivan_old

darn still a failure

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

when we deploy an app to the cluster, we usually create a helmfile in the same repo

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
cloudposse/example-app

Example application for CI/CD demonstrations of Codefresh - cloudposse/example-app

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

we call it Helm Cartridge

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

the helmfile deploys the pull-secret chart and then the app chart

oscarsullivan_old avatar
oscarsullivan_old

Right.. had this fundamental misunderstanding:

Kubernetes deployments are based on a “pull” approach. When you deploy your application to a Kubernetes cluster you don’t upload the application itself (which usually happens with traditional deployments). Instead, Kubernetes will pull the Docker images to its nodes on its own.

https://codefresh.io/docs/docs/deploy-to-kubernetes/access-docker-registry-from-kubernetes/

Accessing a Docker registry from your Kubernetes cluster

Allowing Kubernetes to pull Docker images from your registry

oscarsullivan_old avatar
oscarsullivan_old

My understanding was that since I’ve linked Codefresh and EKS and Codefresh + Codefresh registry that was enough??

mgrube avatar
mgrube

I went through this a little while ago, I created a secret in the namespace I was deploying to

kubectl create secret docker-registry ${name_of_secret} --docker-server=[r.cfcr.io](http://r\.cfcr\.io) --docker-username=${codefresh_username} --docker-password=${codefresh_cr_generated_password} -n ${namespace}

Then when you deploy you have to set the imagePullSecret in the Deployment spec

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

k8s itself needs permissions to pull images

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

(that’s for @oscarsullivan_old ^)

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

when pulling the image during deployment

oscarsullivan_old avatar
oscarsullivan_old

These docs imply this gives permissions

mgrube avatar
mgrube

yeah thats the pull secret that you upload as a secret to kubernetes

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

it has nothing to do with Codefresh, which just sends k8s resource definitions to the cluster, the cluster itself pulls the images

oscarsullivan_old avatar
oscarsullivan_old

I went through this a little while ago, I created a secret in the namespace I was deploying to

kubectl create secret docker-registry ${name_of_secret} --docker-server=[r.cfcr.io](http://r\.cfcr\.io) --docker-username=${codefresh_username} --docker-password=${codefresh_cr_generated_password} -n ${namespace}

Then when you deploy you have to set the imagePullSecret in the Deployment spec

oscarsullivan_old avatar
oscarsullivan_old

It’s just From now on, this cluster on this namespace will be able to deploy Docker images from the selected Registry. sort of says what you’re saying @Andriy Knysh (Cloud Posse)

oscarsullivan_old avatar
oscarsullivan_old


the cluster itself pulls the images

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

yes, it does the same as the chart

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

(too many moving parts )

oscarsullivan_old avatar
oscarsullivan_old

so if that does the same as the chart

oscarsullivan_old avatar
oscarsullivan_old

need I do Mark’s step?

oscarsullivan_old avatar
oscarsullivan_old

and need I do what you’re referencing needs doing?

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

For a quick test, do what Mark showed

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

We use the chart and the helmfile to automate all of that

oscarsullivan_old avatar
oscarsullivan_old


–docker-password=${codefresh_cr_generated_password}
Is this the token you can generate or my actual codefresh password???

mgrube avatar
mgrube

thats the token

oscarsullivan_old avatar
oscarsullivan_old

and the username is the name of the token

oscarsullivan_old avatar
oscarsullivan_old

or my username

mgrube avatar
mgrube

its whatever you set here

mgrube avatar
mgrube

right now it would be your username, unless you have a service account

oscarsullivan_old avatar
oscarsullivan_old

fab

mgrube avatar
mgrube

in the future I think they are going to allow you to generate users for CFCR that are not tied to an individual user

oscarsullivan_old avatar
oscarsullivan_old

yeh makes sense

oscarsullivan_old avatar
oscarsullivan_old

once you added that to the cluster’s namespace with kubectl, did you need to referene the secret?

mgrube avatar
mgrube

yup, I referenced it in the helm chart

mgrube avatar
mgrube

in the Deployment spec

oscarsullivan_old avatar
oscarsullivan_old

Ah damn I don’t have any helm bits setup

oscarsullivan_old avatar
oscarsullivan_old

Should I get familiar with that via kubernetes.io ‘s tutorial?

oscarsullivan_old avatar
oscarsullivan_old


There is nothing specific to Codefresh regarding the usage of Docker registry secrets, and therefore following the official Kubernetes documentation is the recommended approach.
Yep ok

mgrube avatar
mgrube

if you are not using helm you can just set it in the deployment spec itself https://kubernetes.io/docs/concepts/containers/images/

Images attachment image

If you need access to multiple registries, you can create one secret for each registry. Kubelet will merge any imagePullSecrets into a single virtual .docker/config.json

mgrube avatar
mgrube
cat <<EOF > pod.yaml
apiVersion: v1
kind: Pod
metadata:
  name: foo
  namespace: awesomeapps
spec:
  containers:
    - name: foo
      image: janedoe/awesomeapp:v1
  imagePullSecrets:
    - name: myregistrykey
EOF
mgrube avatar
mgrube

here is an example for a pod from their documentation

mgrube avatar
mgrube

myregistrykey would be the name of the secret you created in the namespace

oscarsullivan_old avatar
oscarsullivan_old

Can I execute an ansible playbook in a code fresh pipeline?

oscarsullivan_old avatar
oscarsullivan_old

Instead of K8s deployment it pulls the latest container with ansible. K8s doesn’t feel in scope for the nest 3 months after having tried it for a day it is clear it is a 6-9 month project

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Yes you can execute anything that fits in a container

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

I think geodesic has ansible too :-)

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

So just run geodesic as part of your Codefresh pipeline

oscarsullivan_old avatar
oscarsullivan_old

Ooo wow ok that saves me a lot!

oscarsullivan_old avatar
oscarsullivan_old

Hadn’t considered using my module lol

oscarsullivan_old avatar
oscarsullivan_old

Will need to start actually pushing that up now / placing it in a CI pipeline

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

That’s the beauty of it! It’s a container. Run it anywhere containers are run… locally or in CICD. Same workflow.

oscarsullivan_old avatar
oscarsullivan_old
Freestyle

Run commands inside a Docker container

2019-03-27

casey avatar
casey

has anyone used chamber in a codefresh pipeline to inject secrets into helm chart? Wondering whats that best approach for this?

casey avatar
casey

never mind found a link in the docs, here is reference for anyone who cares

2019-03-25

dustinvb avatar
dustinvb

Codefresh tackles the move from VMs to micro-services supporting build, test, delivery and deployment of micro-services using micro-service based pipelines built upon a scalable platform meant to scale as micro-services emerge which require 10x-1000x the amount of builds in a CI/CD platform as your typical Legacy build VM based applications. Each micro-service can bring in it’s own version of a programming language (best for the job) leaving it up to a VM to build and test offers little flexibility to remain as agile as today’s containers allow. Using a pipeline built out of micro-services gives your CI/CD the same flexibility as the applications you’re building. Around all of this Codefresh has build a UI that integrates with your Version Control System and Kubernetes. Giving you a soup to nuts view of your CI/CD around containers and Kubernetes. We also provide private Docker registry to aid in your archive/deployment of Docker containers. Please see https://steps.codefresh.io/ for a listing of some of the provides micro-services you can include as steps in your pipeline and of course to this document about Codefresh pipelines: https://codefresh.io/docs/docs/configure-ci-cd-pipeline/introduction-to-codefresh-pipelines/

Codefresh: FreshSteps

One small step for your YAML, one giant leap for your pipeline.

Introduction to Codefresh pipelines

Understand how Codefresh pipelines work

2019-03-23

Avineshwar Pratap Singh avatar
Avineshwar Pratap Singh

Noob question: Hoping to understand (read if there is a resource already) the motivation behind creating *codefresh*. Just to be clear, this is not an ask what is *codefresh* but more around the lines of what kind of issues it tackles better.

@Erik Osterman (Cloud Posse)

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@Avineshwar Pratap Singh can you ping me on Monday? I am AFK.

Avineshwar Pratap Singh avatar
Avineshwar Pratap Singh

@Erik Osterman (Cloud Posse) looking forward!

2019-03-21

dustinvb avatar
dustinvb

I already booked the time to attend! This is a do not miss in my book.

1
1

2019-03-20

oscarsullivan_old avatar
oscarsullivan_old

https://codefresh.io/docs/docs/getting-started/create-a-basic-pipeline/#running-unit-tests-automatically Is this still valid? Can’t seem to find the ‘Workflow’–>‘Unit Test’ screen.

Getting Started - Create a Basic Pipeline

Continuous Integration with Codefresh

oscarsullivan_old avatar
oscarsullivan_old

Ah, I had ‘YAML’ mode on

oscarsullivan_old avatar
oscarsullivan_old

I am noticing that I’m unable to ‘Launch’ when using YAML mode? I get a warning confirming this isn’t supported.

oscarsullivan_old avatar
oscarsullivan_old

Alright switched to BASIC mode. It defaulted to YAML, but they seem to do the same things! .. do prefer to have it source vontrolled…. feel like I’ve missed a step

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

you can definitely have it source controlled

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

3 ways to define it:

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
  • 1) inline using UI (great for debugging, but not as a workflow)
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
  • 2) from a remote URL
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
  • 3) from source control (how we typically do it)
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

you can also even define your pipeline creation process as code as we do here:

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
cloudposse/codefresh

Codefresh repos and pipelines in code. Contribute to cloudposse/codefresh development by creating an account on GitHub.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
03:04:05 AM
superfresh1
1

2019-03-19

Codefresh Status - Incident History avatar
Codefresh Status - Incident History
03:54:49 PM

Codefresh Incident Mar 19, 15:53 UTC Investigating - We are investigating an issue on our platform that is affecting build functionality. We will go to provide more information here as investigation progresses.

Codefresh Incident

Codefresh’s Status Page - Codefresh Incident.

oscarsullivan_old avatar
oscarsullivan_old

For Codefresh, builds are effectively PRs aren’t they? As well as daily/weekly builds?

oscarsullivan_old avatar
oscarsullivan_old

Surely 220/m would go v quickly

oscarsullivan_old avatar
oscarsullivan_old

just trying to work that out

Codefresh Status - Incident History avatar
Codefresh Status - Incident History
04:29:54 PM

Codefresh Incident Mar 19, 16:15 UTC Monitoring - There was an issue during a DB upgrade process. The issue was quickly identified and now is resolved. But we’ll keep monitoring the platform to ensure it is working as expected Mar 19, 15:53 UTC Investigating - We are investigating an issue on our platform that is affecting build functionality. We will go to provide more information here as investigation progresses.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
dustinvanbuskirk/github-commit-statistics

Small script to gather github commit statistics. Contribute to dustinvanbuskirk/github-commit-statistics development by creating an account on GitHub.

dustinvb avatar
dustinvb

This should give you a semi reasonable estimate based on commits to your org repos and the total across all of them.

oscarsullivan_old avatar
oscarsullivan_old

Brill thanks. Just had a discussion about moving away from jenkins

oscarsullivan_old avatar
oscarsullivan_old

Just weighing up the costs (setup/migration + long term operating costs) of other platforms.

oscarsullivan_old avatar
oscarsullivan_old

Jenkins currently costing us $84/m (t2.large + EBS) for 2 concurrent builds… However it is a ball ache to do properly.

oscarsullivan_old avatar
oscarsullivan_old

The maintenance cost alone is huge

dustinvb avatar
dustinvb

Yes, our SaaS model gets you out of maintenance and gives you a team of support engineers on our side to assist you if you do run into anything.

dustinvb avatar
dustinvb

I encourage you to reach out to [email protected] to ask any questions about pricing.

oscarsullivan_old avatar
oscarsullivan_old

Just had a count of historic builds on Jenkins

oscarsullivan_old avatar
oscarsullivan_old

we only just breached 60 builds last month

oscarsullivan_old avatar
oscarsullivan_old

and only on 35 builds 19 days into march

dustinvb avatar
dustinvb
Jenkins Plugins

Jenkins – an open source automation server which enables developers around the world to reliably build, test, and deploy their software.

oscarsullivan_old avatar
oscarsullivan_old

turns out we should be fine on the Basic plan

dustinvb avatar
dustinvb

I haven’t had much success with it.

oscarsullivan_old avatar
oscarsullivan_old

no just manually counted lol

dustinvb avatar
dustinvb

OH well… If you can manually count then probably wouldn’t even be a metric to concern yourself with. Instead look at concurrency and the runtime sizes memory/compute for each pipeline.

oscarsullivan_old avatar
oscarsullivan_old

Having come into my org and not having setup the Jenkins and knowing it isn’t properly backed up, I wouldnt even dare install that plugin Dustin!

oscarsullivan_old avatar
oscarsullivan_old

That’s why I made a point here at work that we should move away

oscarsullivan_old avatar
oscarsullivan_old

I shouldn’t feel scared of infrastructure!

dustinvb avatar
dustinvb

Jenkins is a bit brittle.

oscarsullivan_old avatar
oscarsullivan_old

If I’m honest, I COULD quite easily have a Dockerised Jenkins with a mounted EBS volume that gets backed up……. but I just don’t think I want to

oscarsullivan_old avatar
oscarsullivan_old

I would rather we used a SaaS CI tool like CircleCI/Travis/Cloud Fresh

oscarsullivan_old avatar
oscarsullivan_old

But at the same time it would be quite hard to have this dockerised jenkins scale to use slave workers etc

oscarsullivan_old avatar
oscarsullivan_old

and then restore the config when updating the EC2

dustinvb avatar
dustinvb

I’ve done it. Terraform + Chef and then ran all the builds from Docker. It was better but I still had to deal with things like the Maven plugin…

oscarsullivan_old avatar
oscarsullivan_old

if I commit to Codefresh I have to commit to dockerising all our products hmmmm

dustinvb avatar
dustinvb

Not true

oscarsullivan_old avatar
oscarsullivan_old

It would feel like an anti-pattern to dockerise all our applications, have CI run on the dockerised version, but not deploy it as a container

dustinvb avatar
dustinvb

Easy to roll in anything you’re doing on the Linux Slave with Jenkins over to an Ubuntu or CentOS Docker image and incorporate that same scripting and push binaries. That’s supported.

oscarsullivan_old avatar
oscarsullivan_old

Oh? That does sound doable

oscarsullivan_old avatar
oscarsullivan_old

Yes.. interesting

dustinvb avatar
dustinvb

Depends, we have customers that push jars for example that are used by their customers that are not focused on the Docker artifact. I think it depends on what your plan is for the artifact. If it makes sense to run in a container and it is possible that’s an optin.

oscarsullivan_old avatar
oscarsullivan_old

I literally could use our same pipeline but in codefresh

dustinvb avatar
dustinvb

Yup, and if you’re like me you already have the scripting in version control in something like a make file.

oscarsullivan_old avatar
oscarsullivan_old

I do not, I have been here for just over a month and desperately want it out of a platform

oscarsullivan_old avatar
oscarsullivan_old

Practically unreadable in Jenkins (not using Blueocean or Jenkinsfile)

dustinvb avatar
dustinvb

Just leaving the item of building the slave Docker image by running apt-get and other package solutions to prepare the image for the build. Then you can use that in a multistage later on.

oscarsullivan_old avatar
oscarsullivan_old

Does Codefresh have a CF vs other page?

oscarsullivan_old avatar
oscarsullivan_old

I..e vs Travis and Circle

dustinvb avatar
dustinvb

Yes we have several

dustinvb avatar
dustinvb
Continuous Delivery Blog - Codefresh

Learn how to achieve continuous delivery to Kubernetes with Helm, Istio, Selenium testings, security testing, performance testing, and more.

dustinvb avatar
dustinvb

Check the blog here.

oscarsullivan_old avatar
oscarsullivan_old

Thanks, will search in that

dustinvb avatar
dustinvb

@Kostis (Codefresh) Is our technical writer and does the comparisons posted on the blog if you have any questions.

oscarsullivan_old avatar
oscarsullivan_old

Haha no luck using the search bar for “Travis” and “Circle”

dustinvb avatar
dustinvb
Migrate from TravisCI to Codefresh - The New Stack attachment image

TravisCI was recently bought by Idera. And it seems that this acquisition, with reported layoffs, is  encouraging people to look for alternatives. Here’sa tutorial on how to use Coefresh as an alternative.

oscarsullivan_old avatar
oscarsullivan_old

Does it count as a build if I create a PR, build triggered, and I then update the PR?

oscarsullivan_old avatar
oscarsullivan_old

Is that 1 or 2 builds?

oscarsullivan_old avatar
oscarsullivan_old

Say first build fails

oscarsullivan_old avatar
oscarsullivan_old

and then I fix it

dustinvb avatar
dustinvb

2 builds, but could restrict this to not create on specific events.

oscarsullivan_old avatar
oscarsullivan_old

Is the notion of a build applicable when I’m just testing? I figure yes, as in order to test it must build in advance

dustinvb avatar
dustinvb

Say if you have filter on commit and PR open, the commit would have already built unless you wanted to do more in regards to a commit pipeline vs. pr pipeline. Then you’d add pull request synchronized event to build on your commits to existing PR.

oscarsullivan_old avatar
oscarsullivan_old

Alright thanks.. quite a niche scenario from what I’ve seen.. Devs generally test in advance of PR

oscarsullivan_old avatar
oscarsullivan_old

but just wanted to know whether it is easy to burn through build count

dustinvb avatar
dustinvb

Oscar, I’d suggest emailing Michael about the pricing. He can work with you in regards to concerns around these soft build limits in our plans and find you a plan that fits your budget.

oscarsullivan_old avatar
oscarsullivan_old

Thanks Dustin. I did mean to reply actually but got caught up elsewhere.

dustinvb avatar
dustinvb
05:04:14 PM
oscarsullivan_old avatar
oscarsullivan_old

I swear I’m not a busy-bod

oscarsullivan_old avatar
oscarsullivan_old

I do really do work

dustinvb avatar
dustinvb
05:06:08 PM
dustinvb avatar
dustinvb

Not you ?

oscarsullivan_old avatar
oscarsullivan_old

No.. more like

oscarsullivan_old avatar
oscarsullivan_old
05:06:44 PM
dustinvb avatar
dustinvb

Oscar, Michael has confirmed there is alternative plans he can discuss with you. Just FYI.

oscarsullivan_old avatar
oscarsullivan_old

Thanks, have replied to the existing email chain

Codefresh Status - Incident History avatar
Codefresh Status - Incident History
07:25:28 PM

Codefresh Incident Mar 19, 19:11 UTC Update - We’re still working on resolving some performance issues. We’re constantly monitoring the system. Once all minor issues are fully resolved, this incident will be closed Mar 19, 16:15 UTC Monitoring - There was an issue during a DB upgrade process. The issue was quickly identified and now is resolved. But we’ll keep monitoring the platform to ensure it is working as expected Mar 19, 15:53 UTC Investigating - We are investigating an issue on our platform that is…

Codefresh Incident

Codefresh’s Status Page - Codefresh Incident.

Codefresh Status - Incident History avatar
Codefresh Status - Incident History
07:55:13 PM

Codefresh Incident Mar 19, 19:46 UTC Resolved - The issue is now resolved. Performance is now at expected levels across all the platform. We’re going to keep monitoring the system to ensure a consistent behavior in terms of performance Mar 19, 19:37 UTC Update - UPDATE: we’re currently performing a DB-upgrade operation as part of the actions to resolve the current performance issues Mar 19, 19:11 UTC Update - We’re still working on resolving some performance issues. We’re constantly monitoring the system. Once…

2019-03-15

dustinvb avatar
dustinvb

Simple Step for Aqua scanning in a Codefresh pipeline. https://github.com/codefresh-contrib/cfstep-aqua

codefresh-contrib/cfstep-aqua

Codefresh Step for Aqua Docker Image Scans. Contribute to codefresh-contrib/cfstep-aqua development by creating an account on GitHub.

dustinvb avatar
dustinvb
04:19:17 PM

This is the results passed back to Codefresh registry and there is a link to the detailed report in Aqua

:--1:2
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@dustinvb did something change in the last 2-3 weeks in codefresh that would lead to this to start failing?

chsh -s /bin/sh nobody
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

got it

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

something in alpine:3.7 has changed

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

so that it prompts for a password

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

but doesn’t happen in 3.8

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

i’ll figure this out.

dustinvb avatar
dustinvb

I normally see this error related to npm install using global flag. Not sure why it’s going to fail here. I have to add the unsafe arg to npmconfig to get around it.

oscarsullivan_old avatar
oscarsullivan_old

+

oscarsullivan_old avatar
oscarsullivan_old

When I use -g for npm it expects sudo permissions. This is when using node-alpine

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
chsh always asking a password , and get `PAM: Authentication failure`

Today I tried to switch to another shell. First I tried fish, and used chsh -s fish to change fish to default. After some time I found it cannot use ~/.bashrc (&& needs to be replaced by a…

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

this worked for me

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

echo "auth sufficient pam_shells.so" > /etc/pam.d/chsh in my alpine container

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@dustinvb can i unexport an env? (E.g. cf_unexport)

dustinvb avatar
dustinvb

Sorry no, not a feature at this time.

2019-03-14

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Just incase anyone is planning on rolling out Codefresh Enterprise (on prem), in AWS, here’s our terraform module for it: https://github.com/cloudposse/terraform-aws-codefresh-backing-services

cloudposse/terraform-aws-codefresh-backing-services

Terraform module to provision AWS backing services necessary to run Codefresh Enterprise - cloudposse/terraform-aws-codefresh-backing-services

superfresh1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

this is to leverage as many fully-managed AWS services as possible that are required to run Codefresh

2019-03-13

Codefresh Status - Incident History avatar
Codefresh Status - Incident History
09:00:31 AM

GitHub Incident - Degraded performance across Web, Pages and Notifications Mar 13, 08:51 UTC Resolved - This incident has been resolved. Mar 12, 15:18 UTC Investigating - GitHub has reported an incident related to degraded performance across Web, Pages and Notifications. More information here: https://www.githubstatus.com/

GitHub Incident - Degraded performance across Web, Pages and Notifications

Codefresh’s Status Page - GitHub Incident - Degraded performance across Web, Pages and Notifications.

GitHub Status

Welcome to GitHub’s home for real-time and historical data on system performance.

dustinvb avatar
dustinvb

Sorry been traveling see you’re already here @oscarsullivan_old

oscarsullivan_old avatar
oscarsullivan_old

Indeed! Looking forward to spending tomorrow using Codefresh. Thanks for the demo-call Dustin!

2

2019-03-12

Codefresh Status - Incident History avatar
Codefresh Status - Incident History
03:20:25 PM

GitHub Incident - Degraded performance across Web, Pages and Notifications Mar 12, 15:18 UTC Investigating - GitHub has reported an incident related to degraded performance across Web, Pages and Notifications. More information here: https://www.githubstatus.com/

GitHub Incident - Degraded performance across Web, Pages and Notifications

Codefresh’s Status Page - GitHub Incident - Degraded performance across Web, Pages and Notifications.

GitHub Status

Welcome to GitHub’s home for real-time and historical data on system performance.

oscarsullivan_old avatar
oscarsullivan_old

How come the github status comes to this channel @Erik Osterman (Cloud Posse)?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

The Codefresh status page includes statuses of everything they depend on

oscarsullivan_old avatar
oscarsullivan_old

Ah ok, cool

oscarsullivan_old avatar
oscarsullivan_old

I just subscribed to github status rss for my firm’s slack.. awesome idea.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Yea it’s worked wonderfully! We have a channel at CloudPosse with feeds for all the status pages (including AWS) that we depend on

2019-03-08

dustinvb avatar
dustinvb
Git providers

Codefresh is a Docker-native CI/CD platform. Instantly build , test and deploy Docker images.

2

2019-03-06

dustinvb avatar
dustinvb
Access control

How to restrict resources in a company environment

party_parrot2

2019-03-04

mgrube avatar
mgrube

has anyone ever set up deploy conditionals to only deploy when a branch is tagged?

dustinvb avatar
dustinvb

Yes, Mark I have.

I normally evaluate that the variable ${{CF_BASE_BRANCH}} is set to master

This will have a value on tags.

Real world example.

when:
      condition:
        all:
          releaseTag: 'match("${{CF_BRANCH}}", "^([0-9]+)([.][0-9]+){1,2}.*", true) == true'
          masterBranch: '"${{CF_BASE_BRANCH}}" == "master"'
dustinvb avatar
dustinvb

This will look at branch for semantic version as tag and also make sure the branch used was master not some feature.

:--1:1
mgrube avatar
mgrube

you rock, dustin

mgrube avatar
mgrube

It looks like CF_BASE_BRANCH does not exist on Bitbucket? Do you know any other way to pull the name of the branch, not the tag? even when I do

    when:
      condition:
        all:
          releaseTag: 'match("${{CF_BRANCH}}", "^([0-9]+)([.][0-9]+){1,2}.*", true) == true'
      branch:
        only:
          - master

the branch still gets pulled as the tag eg 1.0.0

mgrube avatar
mgrube

when I do a build with echo "CF_BASE_BRANCH=${{CF_BASE_BRANCH}}", it says there is bad variable substitution

dustinvb avatar
dustinvb

During the event we pick up tag and use for CF_BRANCH based on the event. I am not sure why BASE_BRANCH is not available. Please send this through Codefresh’s Intercom to be tracked by support. BTW new events for Bitbucket have been released providing more filtering options around PRs.

:--1:1

2019-03-01

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@dustinvb is there are way to update shared configurations using cli?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

thx!!

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

feels very much like kubectl

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

too bad it doesn’t support -f - for stdin

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
Error: Failed to read file; caused by Error: File extension is not recognized

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@dustinvb got what I wanted to implement (assume roles) for codefresh

    keyboard_arrow_up