#codefresh (2019-05)
Archive: https://archive.sweetops.com/codefresh/
2019-05-01

whats the new unlimited builds pricing model for codefresh going to be like?

@oriel can help

@oriel has joined the channel

for additional context see #gitops

@btai we’re looking to have the model which will be per concurrent steps, so no matter how many build you’re running overall it’s all about the concurrency of steps. Are you looking for specific pricing? Or just general idea of how it will work?

Curious what the specific pricing will be
2019-05-02

When are you guys going to roll out the new pricing schema @oriel?



hahah! love this image

I made this a slack emoji a while back, some of my developers REALLY hate it, others love it

In case anyone missed it (New Features Webinar): https://www.youtube.com/watch?v=z3u5cCV7glo&feature=youtu.be

How do you turn off the new projects feature…. It is breaking my ability to work with and launch pipelines

@Michael Kolb Can you send this information to [email protected] along with some additional details around what items are no longer working after the Project Features implementation? Thank you.
2019-05-04

Docker Hub Planned Maintenance THIS IS A SCHEDULED EVENT May 4, 16:00 UTC - May 5, 02:15 UTC May 4, 15:04 UTC Scheduled - Docker Hub will be performing scheduled maintenance this Saturday May 4, 2019 from approximately 9:00AM to 7:15PM, US Pacific Daylight Time (UTC-7). During this window, Docker Hub will be operating in a read-only mode. Registry logins and image pulls will continue to work for a majority of this time frame. Pushes however will generally be unavailable. Maintenance activities, approximate timelines…
Codefresh’s Status Page - Docker Hub Planned Maintenance.

Docker Hub Planned Maintenance May 4, 16:00 UTC In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary. May 4, 15:04 UTC Scheduled - Docker Hub will be performing scheduled maintenance this Saturday May 4, 2019 from approximately 9:00AM to 7:15PM, US Pacific Daylight Time (UTC-7). During this window, Docker Hub will be operating in a read-only mode. Registry logins and image pulls will continue to work for a majority of this time frame. Pushes however will generally be…

Docker Hub Planned Maintenance May 5, 02:16 UTC Completed - The scheduled maintenance has been completed. May 4, 16:00 UTC In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary. May 4, 15:04 UTC Scheduled - Docker Hub will be performing scheduled maintenance this Saturday May 4, 2019 from approximately 9:00AM to 7:15PM, US Pacific Daylight Time (UTC-7). During this window, Docker Hub will be operating in a read-only mode. Registry logins and image pulls will continue to work…
Codefresh’s Status Page - Docker Hub Planned Maintenance.
2019-05-06

codefresh.io website is down May 6, 16:00 UTC Monitoring - Consumer website is back up. We are monitoring. May 6, 15:59 UTC Update - We are continuing to work on a fix for this issue. May 6, 15:56 UTC Identified - Our consumer facing website is down but you can still log into your services at https://g.codefresh.io
Our hosting provider for our consumer facing site is looking into server issues.
Codefresh’s Status Page - codefresh.io website is down.

codefresh.io website is down May 6, 16:36 UTC Resolved - This incident has been resolved. May 6, 16:00 UTC Monitoring - Consumer website is back up. We are monitoring. May 6, 15:59 UTC Update - We are continuing to work on a fix for this issue. May 6, 15:56 UTC Identified - Our consumer facing website is down but you can still log into your services at https://g.codefresh.io
Our hosting provider for our consumer facing site is looking into server issues.
2019-05-07

Build initialization issue May 7, 10:50 UTC Resolved - We had experienced a 30 minutes issue which affected initializing new builds. 2:51 am - 3:19 am PDT
Codefresh’s Status Page - Build initialization issue.
2019-05-10

How do I go about launching two containers (a la docker-compose) in a pipeline? I run GOSS in my pipeline but it depends on access to a database which I’d like to be inside the pipeline instead of a real one

You will need to run docker compose in daemon mode.

so that execution can continue and you can do testing.

the database should be inside the docker-compose

and you will need a waiting script.

what I do is stick this in the docker-compose for the app
command:
- /bin/sh
- -c
- |
while ! nc -z mysql-test 3306; do
echo "Database is not ready yet."
sleep 1
done

mysql-test
is the name of the service in docker-compose

does this help ?

also -exit-code-from
is helpful https://docs.docker.com/compose/reference/up/
Usage: up [options] [–scale SERVICE=NUM…] [SERVICE…] Options: -d, –detach Detached mode: Run containers in the background, print new container names. Incompatible with –abort-on-container-exit. –no-color Produce monochrome output. –quiet-pull Pull without…
2019-05-12

Codefresh UI doesn’t load May 12, 16:02 UTC Investigating - We are currently investigating an issue where Codefresh UI doesn’t load. Build automatic triggers are not affected.
Codefresh’s Status Page - Codefresh UI doesn’t load.

Codefresh UI doesn’t load May 12, 16:20 UTC Resolved - The incident has been resolved May 12, 16:02 UTC Investigating - We are currently investigating an issue where Codefresh UI doesn’t load. Build automatic triggers are not affected.
2019-05-15

@Olivier Vanden Bos please join me here. We run this community Slack in partnership with Cloud Posse.

@Olivier Vanden Bos has joined the channel


doing public office hours now: https://zoom.us/j/684901853

I wish I’d seen this

Every Wednesday at 11:30 am pst


In my calender weekly!

if anyone has questions on codefresh…

Can I invite some of my developers to this slack channel ?

please do - it’s open for everyone

https://slack.sweetops.com <- signup

Question About pushing images to dockerhub

I usually do that locally to push : docker tag game-client imbellus/game-client-build:1 docker push imbellus/game-client-build:1

Error in code fresh is : Running push step: Pushing to Docker Registry Pulling image ovandenbos-imb/game-client18865fc9fba8ac10129b14677adfbac370858d320c646fa7697ad46791b9f90a Digest: sha256:18865fc9fba8ac10129b14677adfbac370858d320c646fa7697ad46791b9f90a Status: Image is up to date for [r.cfcr.io/ovandenbos-imb/game-client@sha256//r.cfcr.io/ovandenbos-imb/game-client@sha256:18865fc9fba8ac10129b14677adfbac370858d320c646fa7697ad46791b9f90a) Finished pulling [r.cfcr.io/ovandenbos-imb/game-client@sha256//r.cfcr.io/ovandenbos-imb/game-client@sha256:18865fc9fba8ac10129b14677adfbac370858d320c646fa7697ad46791b9f90a) from internal registry Tagging image as: [docker.io/game-client//docker.io/game-client:1) The push refers to repository [docker.io/library/game-client] denied: requested access to the resource is denied [SYSTEM] Error: Failed to push image [docker.io/game-client//docker.io/game-client:1;) caused by NonZeroExitCodeError: Container for operation: push failed with exit code: 12

for the yaml: client_image: title: Building Docker Image type: build stage: build image_name: game-client working_directory: ./ tag: ‘${{CF_BRANCH_TAG_NORMALIZED}}’ dockerfile: Dockerfile
push_to_registry: title: Pushing to Docker Registry type: push stage: postbuild candidate: ‘${{client_image}}’ tag: ‘1’ registry: dockerhub

any idea of what I am doing wrong ?

‘dockerhub ‘ registry is setup and ‘Test’ successful

the obvious step that I am missing is the ‘rename’ part in tagging : docker tag game-client imbellus/game-client-build

The image_name
needs to be relative to DockerHub

e.g. for us, it would be cloudposse/image-client
since cloudposse
is our org

Here’s how we do it

Geodesic is a cloud automation shell. It's the fastest way to get up and running with a rock solid, production grade cloud platform built on top of strictly Open Source tools. ★ this repo! h…

this works because our CF_REPO_OWNER
(cloudposse) is the same as our dockerhub

Cool. lemme try.

It worked thank

Freestyle npm step stuck

my log looks like that :

the page is actually freezing and I cannot copy to the clipboard, but that’s a different issue

this works locally for me

maybe because we are using a private repo on git ?

“dependencies”: { “chart.js”: “^2.8.0”, “chartjs-plugin-datalabels”: “^0.5.0”, “detect-browser”: “^4.4.0”, “gameui-design-system”: “<git+ssh://[email protected]/ImbellusBetta/gameui-design-system.git#v3.7.3>”, “lodash.debounce”: “^4.0.8”, “math3d”: “^0.2.2”, “object-path”: “^0.11.4”,

@dustinvb can you please give this one a look when you get a chance?

@Olivier Vanden Bos Can you send me the Build’s URL from the logs page you pulled the excerpt. I’d like to examine the commands in more detail there is a possibility that we need pass credentials to the step to allow access to private repository.

I see this is using the ssh method to clone the private repository. I believe this is a Docker build process. I am checking now to see how we can get the context you’ve created and convert that to be able to specify a SSH key.

I think we will need to incorporate something like the following into the Dockerfile itself using a combination of build arg to import the key to be used. Do you have a service account user that you can create an ssh key for that has access to the npm repository?

Sorry @dustinvb this one I fixed. Found the doc and that fixed it. In deed, the ‘Freeze’ was really annoying and lack of logs (but could get better logs with the download panel)

If you see this freeze again can you please send me the build’s url and the step and spot where the freeze occurred. I’d like to have support look. If you have the original I wouldn’t mind looking at that.

I can


this one is the latest one

froze for 1h 10m

I might have cancelled it

this one was before. 3h https://g.codefresh.io/build/5cdccad416a7dfc61fe05d4b?step=client_ui&tab=output

Can I get your OS + version and Browser + version?

Windows 10 Chrome Version 74.0.3729.157
2019-05-16

Did you know https://codecov.io/ works with codefresh: https://docs.codecov.io/docs/supported-ci-providers?
I did not realize, since we did code coverage in composition and env variables did not get passed, but after passing these into the composition candidate made it “just work”:
- CODECOV_SLUG=${{CF_REPO_OWNER}}/${{CF_REPO_NAME}}
- CF_BUILD_URL=${{CF_BUILD_URL}}
- CF_BUILD_ID=${{CF_BUILD_ID}}
- CF_BRANCH=${{CF_BRANCH}}
- CF_REVISION=${{CF_REVISION}}
(on top of CODECOV_TOKEN)
Develop healthier code using Codecov’s leading, dedicated code coverage solution. Try it free

^^ we had codecov working on codefresh already, but that just adds extra metadata on the codecov side - like link to build etc.

Thanks for sharing

playing with codefresh, ran into an interesting issue where not including the tag
key for the build step, despite it being marked as having a default value in the docs, was throwing this error TypeError: Cannot read property 'branch' of undefined
during the initialization process. couldnt find anything about this anywhere so hopefully this helps someone
2019-05-17

trying to use the codefresh build-harness to setup pipelines ( https://github.com/cloudposse/build-harness/tree/master/modules/codefresh ), running into an issue Error: Default git context is not specified
. according to the spec, context is optional ( https://codefresh-io.github.io/cli/pipelines/spec/ ), and i cant find a reference to what the “auto select branch” option should be. looking at the yaml from an existing pipeline shows only the same 3 options im trying to provide in the spec template (location, repo, path)

this might help: https://github.com/cloudposse/codefresh
Codefresh repos and pipelines in code. Contribute to cloudposse/codefresh development by creating an account on GitHub.

Running out of resources @dustinvb

here are my current real blockers



Error response from daemon: Error processing tar file(exit status 1): write /GameClient/Assets/Scenes/CoralReefRewardScene/R&D/Scenes/RS-LayerTest.unity: no spac e left on device

@oriel FYI

Okay this is why I put you on enterprise to begin with. I knew you were going to be building in a way that you’d need extra space. I believe these pipelines are around 40GB. Do you have an estimate on how big these Docker images might get?

@francisco Could you assist here and just check this build out? We’re building using unity and the images are going to be very large. I am not sure but I think we’re pressing the storage limit defined in the runtime.

40GB seems a lot

Lemme check what’s our real size on Jenkins

Docker source image with UNity is 7GB

my full local repo that build 3 targets is around 8GB

now are you including the git files themselves ? .git ?

my .git looks like 20Gb

Hmm well that should have fit. Let me check with my support team on this.

General questions about best strategy to generate artifacts ala Jenkins

we used to zip and upload to S3. I did add the S3 integration.

Do you have any example on how to zip and upload ?

We deploy as well as a static website - still on S3 - but not zipped this time. any useful plugin ?

@dustinvb I am planning to use directly the aws s3-cli ? Is there anything simpler ? I do not see in the doc how to use the s3 artifact integration ? should I deal with the credentials in the script itself ?

A plugin to work with s3 buckets. Contribute to codefresh-contrib/s3copy development by creating an account on GitHub.

I am looking for a step to aid with archive now.

For that s3 copy you just need to provide a service account in S3 with bucket access using these variables in the pipeline variables.
AWS_ACCESS_KEY_ID=<id> AWS_SECRET_ACCESS_KEY=<key> AWS_DEFAULT_REGION=<region>
If you plan on using this user over and over you may want to put these variables into a shared config you can import for other builds.
How to keep your pipelines DRY

Sorry haven’t forgot still looking. Steps that often require this have this build in. Trying to find an a-la-carte example.

ArchiveFiles:
image: alpine:latest
commands:
- apk add --update zip
- tar -czvf ./archive.tgz ./folder/or/file.txt
I do not see that we have a step for this.
Please try the step below using an alpine image. I am checking with team about OOTB step.

perfect thanks !


ive traced my issue down to codefresh create -f
failing no matter what i give it, giving me Error: "{\"status\":500,\"code\":\"1001\",\"name\":\"INTERNAL_SERVER_ERROR\",\"message\":\"Internal server error\",\"context\":{}}"
not really sure where to go from here

You go right here… Sorry about the delay. I am traveling and often offline on weekends.
Please send me your YAML file for review. It is possible you’re missing a value or somethign.
I have some recent examples I updated here for comparison.
https://github.com/codefresh-contrib/example-voting-app/tree/master/.codefresh/specs
Docker’s Example Voting App. Contribute to codefresh-contrib/example-voting-app development by creating an account on GitHub.

thanks for the reply! https://gist.github.com/mgeist/163026ccfb89626808ec24ca647dc589 using https://codefresh-io.github.io/cli/pipelines/spec/ as a reference, specifically “Pipeline with a remote spec template brought from a git repository”

Can you please try adding a name to your git context and let me know if that corrects the behavior? https://github.com/codefresh-contrib/example-voting-app/blob/master/.codefresh/specs/cd.yaml#L13
Name can be anything to identify the trigger.
I will make sure the team updates our spec examples if this resolves the issue.
Docker’s Example Voting App. Contribute to codefresh-contrib/example-voting-app development by creating an account on GitHub.

same thing, code 1001, 500 internal server error

Can you update your example yaml with the latest?

Do you have a GIT context configured? Can you add these two fields?
context being the name of your context. If not, at a minimum please try to add the default revision.
Docker’s Example Voting App. Contribute to codefresh-contrib/example-voting-app development by creating an account on GitHub.

# codefresh get contexts
NAME TYPE
CF_HELM_DEFAULT helm-repository
github git.github
and updated script: https://gist.github.com/mgeist/163026ccfb89626808ec24ca647dc589 no dice

finally got to the bottom of this, turns out pipeline names will break if they contain /
. thanks for the assistance and FYI @dustinvb

Didn’t expect that to be the case. I will discuss with the team exposing this as an error on create.

It seems like a step in my codefresh.yml ends and proceeds to the next step before the final command exits

here’s what I’m trying to do:

pull down terraform and unzip the archive

here’s the steps in my yaml file:
steps:
clone_repo:
stage: prepare
type: git-clone
repo: '${{CF_REPO_OWNER}}/${{CF_REPO_NAME}}'
revision: '${{CF_REVISION}}'
git: github
wtf:
stage: prepare
image: alpine:latest
commands:
- 'apk add curl wget unzip file'
- 'ls -l'
- 'curl <https://releases.hashicorp.com/terraform/${{TERRAFORM_VERSION}}/terraform_${{TERRAFORM_VERSION}}_linux_amd64.zip> -o /tmp/terraform.zip'
next_part:
stage: prepare
title: 'hello world'
image: alpine:latest
commands:
- 'ls -lh'
- 'unzip /tmp/terraform.zip'

here’s some things I’ve tried:
- fixing some nice typos
- changing the step where
curl
runs - running
curl
with-s
?

a common error that I get that makes me think the step is ending before the curl
exits is this:
End-of-central-directory signature not found. Either this file is not
a zipfile, or it constitutes one disk of a multi-part archive.

thrown by unzip

hey at least it’s friday tho ¯_(ツ)_/¯

my work around for this was just to put these into a dockerfile and build it. that works really well

I am glad you found a work around. Please note that while we do participate here our Intercom “chat” bubble is staffed 7am-6pm PDT M-F whenever these issues come up please check with the support staff there. We also have hours outside of the ones mentioned.

thanks @dustinvb I’ll use the chat bubble next time
2019-05-20

Hi, I’m trying to get a list of files that changed from current branch off a different branch. e.g. git diff $BRANCH –name-only on a pipeline. Seems that that other branches don’t get fetched so I ran a git fetch $BRANCH, but git asks for credentials. Got ssh key in place as suggested on the documentation but it’s not getting picked up by git. Any ideas on what I’m missing or a workaround for this?


client_ui: title: Building Game UI image: ${{client_image}} stage: build_client working_directory: ${{main_clone}} commands: - mkdir -p ~/.ssh - ssh-keyscan -H github.com >> ~/.ssh/known_hosts - echo “${SSH_KEY}” | tr '”${SPLIT_CHAR}”' ‘\n’ > ~/.ssh/id_rsa - chmod 600 ~/.ssh/id_rsa - ‘/bin/bash docker-build-ui.sh’


thanks Olivier!!!

Cache wiped out on a yml change. is it expected ?

Looking a 2 successive build - 27 min with cache and next one end up to be 2 hours


@dustinvb is it expected ?

literally the only changes on the depot were the yml file of codefresh..

@Olivier Vanden Bos will check and get back to you as soon as possible. This is the same issue you described in the email you sent me right?

They all seem related. It is almost if the codefrsh/volume was shared accross pipelines and concurrent build could collide somehow

I managed to repro locally though - with simulating the volume persistent accross pipelines. I am still digging.

Found the issue, was on our side
2019-05-21
2019-05-22

GitHub Incident: delay in notification deliveries May 22, 17:18 UTC Identified - GitHub is experiencing an incident related to their Notifications. This impacts email and webhook deliveries.
More information here: https://www.githubstatus.com/incidents/nf4gcjtcsqdb
Codefresh’s Status Page - GitHub Incident: delay in notification deliveries.
GitHub’s Status Page - Incident on 2019-05-22 16:56 UTC.

Public/Free Office Hours with Cloud Posse starting now!!

GitHub Incident: delay in notification deliveries May 22, 23:46 UTC Resolved - This incident has been resolved. May 22, 17:18 UTC Identified - GitHub is experiencing an incident related to their Notifications. This impacts email and webhook deliveries.
More information here: https://www.githubstatus.com/incidents/nf4gcjtcsqdb
Codefresh’s Status Page - GitHub Incident: delay in notification deliveries.
GitHub’s Status Page - Incident on 2019-05-22 16:56 UTC.
2019-05-23
2019-05-28
2019-05-29

public #office-hours starting now! join us here: https://zoom.us/j/684901853
2019-05-30

what is happening in “Initializing Process”

just started using codefresh and im getting “TypeError: Cannot read property ‘branch’ of undefined”

adding a tag key to the build step fixed that error for me

can you share your yaml as a snippet?

version: '1.0'
stages:
- prepare
- build
steps:
main_clone:
title: Cloning repo
type: git-clone
repo: '${{CF_REPO_OWNER}}/${{CF_REPO_NAME}}'
revision: '${{CF_REVISION}}'
stage: prepare
git: github
build_docker_image:
title: Building Docker Image
type: build
stage: build
image_name: bug-tracker
working_directory: ./
dockerfile: Dockerfile

haha

adding tag to build_docker_image

aha! yes

@osterman Ran into an issue where pipeline would fail to initialize due to the tag argument not being present in the build step. Can you please review? I used the SHORT SHA for the tag.

do we get docker layer caching automatically w/codefresh

yes

the ui is so nice


example of layer caching working

i see a borken image

in slack?


wierd

yeah slack does that sometimes.. refreshing or clicking will show it

if you click on it, it shows

i see it now

do you guys use the codefresh image registry too?

yep

we use that and ECR

for simpler setups without the requirements of ECR use the codefresh registry

ECR is trickier plus max 1000 tags per repo

(which means you cannot use same repo for prod and ci)

i see

bout to test out this layer caching

circle charges for it and its super fickle (barely ever works). but i believe codefresh has a persistent volume per account

PV per pipeline

i believe

@dustinvb

amazing

Example application for CI/CD demonstrations of Codefresh - cloudposse/example-app

here are our pipelines to get you started

how come your build_image doesnt require tag

Example application for CI/CD demonstrations of Codefresh - cloudposse/example-app

i think this is a recent thing

none of our pipelines have it, but they started breaking

ah

Yes, very recent. Already contacted my team about this to update in our docs.

And Yes PV per pipeline. When successful volume stored for next pipeline execution./

are we supposed to setup the webhook in gh ourselves or does codefresh do that automatically for us?


i see that, but no webhook was created in my repo

i just created it myself (not sure if the above is a bug or im understanding it incorrectly)

This should be happening automatically. Can you send me the time you setup the GIT Trigger, Trigger Name and the Pipeline you created this trigger on? I am also assuming you’re using GH SaaS?

what do you mean GH Saas?


yeah

So i had a git push that didnt trigger at this time Thu May 30 13:07:03 2019 -0700

thats PST btw

You said the webhook was not under the repository settings?

You had to add manually?

dming you privately

is there a way to resync your github repositories on codefresh? I created a new repo for shared codefresh configs (build, deploy, etc) and I’m trying to select that repo in the dropdown but it doesn’t show up

This should be syncing automatically but we could be seeing some cache at work here. Can you gather your OS and Browser and send that over to our support team using the Intercom Chat Bubble widget?
2019-05-31

is there a way to delete/update helm repos in the codefresh ui?