#codefresh (2019-05)
Archive: https://archive.sweetops.com/codefresh/
2019-05-01
whats the new unlimited builds pricing model for codefresh going to be like?
@oriel can help
@oriel has joined the channel
for additional context see #gitops
@btai we’re looking to have the model which will be per concurrent steps, so no matter how many build you’re running overall it’s all about the concurrency of steps. Are you looking for specific pricing? Or just general idea of how it will work?
Curious what the specific pricing will be
2019-05-02
When are you guys going to roll out the new pricing schema @oriel?
hahah! love this image
I made this a slack emoji a while back, some of my developers REALLY hate it, others love it
In case anyone missed it (New Features Webinar): https://www.youtube.com/watch?v=z3u5cCV7glo&feature=youtu.be
How do you turn off the new projects feature…. It is breaking my ability to work with and launch pipelines
@Michael Kolb Can you send this information to [email protected] along with some additional details around what items are no longer working after the Project Features implementation? Thank you.
2019-05-04
Docker Hub Planned Maintenance THIS IS A SCHEDULED EVENT May 4, 16:00 UTC - May 5, 02:15 UTC May 4, 15:04 UTC Scheduled - Docker Hub will be performing scheduled maintenance this Saturday May 4, 2019 from approximately 9:00AM to 7:15PM, US Pacific Daylight Time (UTC-7). During this window, Docker Hub will be operating in a read-only mode. Registry logins and image pulls will continue to work for a majority of this time frame. Pushes however will generally be unavailable. Maintenance activities, approximate timelines…
Docker Hub Planned Maintenance May 4, 16:00 UTC In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary. May 4, 15:04 UTC Scheduled - Docker Hub will be performing scheduled maintenance this Saturday May 4, 2019 from approximately 9:00AM to 7:15PM, US Pacific Daylight Time (UTC-7). During this window, Docker Hub will be operating in a read-only mode. Registry logins and image pulls will continue to work for a majority of this time frame. Pushes however will generally be…
Docker Hub Planned Maintenance May 5, 02:16 UTC Completed - The scheduled maintenance has been completed. May 4, 16:00 UTC In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary. May 4, 15:04 UTC Scheduled - Docker Hub will be performing scheduled maintenance this Saturday May 4, 2019 from approximately 9:00AM to 7:15PM, US Pacific Daylight Time (UTC-7). During this window, Docker Hub will be operating in a read-only mode. Registry logins and image pulls will continue to work…
2019-05-06
codefresh.io website is down May 6, 16:00 UTC Monitoring - Consumer website is back up. We are monitoring. May 6, 15:59 UTC Update - We are continuing to work on a fix for this issue. May 6, 15:56 UTC Identified - Our consumer facing website is down but you can still log into your services at https://g.codefresh.io
Our hosting provider for our consumer facing site is looking into server issues.
codefresh.io website is down May 6, 16:36 UTC Resolved - This incident has been resolved. May 6, 16:00 UTC Monitoring - Consumer website is back up. We are monitoring. May 6, 15:59 UTC Update - We are continuing to work on a fix for this issue. May 6, 15:56 UTC Identified - Our consumer facing website is down but you can still log into your services at https://g.codefresh.io
Our hosting provider for our consumer facing site is looking into server issues.
2019-05-07
Build initialization issue May 7, 10:50 UTC Resolved - We had experienced a 30 minutes issue which affected initializing new builds. 2:51 am - 3:19 am PDT
2019-05-10
How do I go about launching two containers (a la docker-compose) in a pipeline? I run GOSS in my pipeline but it depends on access to a database which I’d like to be inside the pipeline instead of a real one
You will need to run docker compose in daemon mode.
so that execution can continue and you can do testing.
the database should be inside the docker-compose
and you will need a waiting script.
what I do is stick this in the docker-compose for the app
command:
- /bin/sh
- -c
- |
while ! nc -z mysql-test 3306; do
echo "Database is not ready yet."
sleep 1
done
mysql-test is the name of the service in docker-compose
does this help ?
also -exit-code-from is helpful https://docs.docker.com/compose/reference/up/
2019-05-12
Codefresh UI doesn’t load May 12, 16:02 UTC Investigating - We are currently investigating an issue where Codefresh UI doesn’t load. Build automatic triggers are not affected.
Codefresh UI doesn’t load May 12, 16:20 UTC Resolved - The incident has been resolved May 12, 16:02 UTC Investigating - We are currently investigating an issue where Codefresh UI doesn’t load. Build automatic triggers are not affected.
2019-05-15
@Olivier Vanden Bos please join me here. We run this community Slack in partnership with Cloud Posse.
@Olivier Vanden Bos has joined the channel
doing public office hours now: https://zoom.us/j/684901853
I wish I’d seen this
Every Wednesday at 11:30 am pst
In my calender weekly!
if anyone has questions on codefresh…
Can I invite some of my developers to this slack channel ?
please do - it’s open for everyone
https://slack.sweetops.com <- signup
Question About pushing images to dockerhub
I usually do that locally to push : docker tag game-client imbellus/game-client-build:1 docker push imbellus/game-client-build:1
Error in code fresh is : Running push step: Pushing to Docker Registry Pulling image ovandenbos-imb/game-client18865fc9fba8ac10129b14677adfbac370858d320c646fa7697ad46791b9f90a Digest: sha256:18865fc9fba8ac10129b14677adfbac370858d320c646fa7697ad46791b9f90a Status: Image is up to date for [r.cfcr.io/ovandenbos-imb/game-client@sha256//r.cfcr.io/ovandenbos-imb/game-client@sha256:18865fc9fba8ac10129b14677adfbac370858d320c646fa7697ad46791b9f90a) Finished pulling [r.cfcr.io/ovandenbos-imb/game-client@sha256//r.cfcr.io/ovandenbos-imb/game-client@sha256:18865fc9fba8ac10129b14677adfbac370858d320c646fa7697ad46791b9f90a) from internal registry Tagging image as: [docker.io/game-client//docker.io/game-client:1) The push refers to repository [docker.io/library/game-client] denied: requested access to the resource is denied [SYSTEM] Error: Failed to push image [docker.io/game-client//docker.io/game-client:1;) caused by NonZeroExitCodeError: Container for operation: push failed with exit code: 12
for the yaml: client_image: title: Building Docker Image type: build stage: build image_name: game-client working_directory: ./ tag: ‘${{CF_BRANCH_TAG_NORMALIZED}}’ dockerfile: Dockerfile
push_to_registry: title: Pushing to Docker Registry type: push stage: postbuild candidate: ‘${{client_image}}’ tag: ‘1’ registry: dockerhub
any idea of what I am doing wrong ?
‘dockerhub ‘ registry is setup and ‘Test’ successful
the obvious step that I am missing is the ‘rename’ part in tagging : docker tag game-client imbellus/game-client-build
The image_name needs to be relative to DockerHub
e.g. for us, it would be cloudposse/image-client since cloudposse is our org
Here’s how we do it
this works because our CF_REPO_OWNER (cloudposse) is the same as our dockerhub
Cool. lemme try.
It worked thank
Freestyle npm step stuck
my log looks like that :
the page is actually freezing and I cannot copy to the clipboard, but that’s a different issue
this works locally for me
maybe because we are using a private repo on git ?
“dependencies”: { “chart.js”: “^2.8.0”, “chartjs-plugin-datalabels”: “^0.5.0”, “detect-browser”: “^4.4.0”, “gameui-design-system”: “<git+ssh://[email protected]/ImbellusBetta/gameui-design-system.git#v3.7.3>”, “lodash.debounce”: “^4.0.8”, “math3d”: “^0.2.2”, “object-path”: “^0.11.4”,
@dustinvb can you please give this one a look when you get a chance?
@Olivier Vanden Bos Can you send me the Build’s URL from the logs page you pulled the excerpt. I’d like to examine the commands in more detail there is a possibility that we need pass credentials to the step to allow access to private repository.
I see this is using the ssh method to clone the private repository. I believe this is a Docker build process. I am checking now to see how we can get the context you’ve created and convert that to be able to specify a SSH key.
I think we will need to incorporate something like the following into the Dockerfile itself using a combination of build arg to import the key to be used. Do you have a service account user that you can create an ssh key for that has access to the npm repository?
Sorry @dustinvb this one I fixed. Found the doc and that fixed it. In deed, the ‘Freeze’ was really annoying and lack of logs (but could get better logs with the download panel)
If you see this freeze again can you please send me the build’s url and the step and spot where the freeze occurred. I’d like to have support look. If you have the original I wouldn’t mind looking at that.
I can
this one is the latest one
froze for 1h 10m
I might have cancelled it
this one was before. 3h https://g.codefresh.io/build/5cdccad416a7dfc61fe05d4b?step=client_ui&tab=output
Can I get your OS + version and Browser + version?
Windows 10 Chrome Version 74.0.3729.157
2019-05-16
Did you know https://codecov.io/ works with codefresh: https://docs.codecov.io/docs/supported-ci-providers?
I did not realize, since we did code coverage in composition and env variables did not get passed, but after passing these into the composition candidate made it “just work”:
- CODECOV_SLUG=${{CF_REPO_OWNER}}/${{CF_REPO_NAME}}
- CF_BUILD_URL=${{CF_BUILD_URL}}
- CF_BUILD_ID=${{CF_BUILD_ID}}
- CF_BRANCH=${{CF_BRANCH}}
- CF_REVISION=${{CF_REVISION}}
(on top of CODECOV_TOKEN)
^^ we had codecov working on codefresh already, but that just adds extra metadata on the codecov side - like link to build etc.
Thanks for sharing
playing with codefresh, ran into an interesting issue where not including the tag key for the build step, despite it being marked as having a default value in the docs, was throwing this error TypeError: Cannot read property 'branch' of undefined during the initialization process. couldnt find anything about this anywhere so hopefully this helps someone
2019-05-17
trying to use the codefresh build-harness to setup pipelines ( https://github.com/cloudposse/build-harness/tree/master/modules/codefresh ), running into an issue Error: Default git context is not specified. according to the spec, context is optional ( https://codefresh-io.github.io/cli/pipelines/spec/ ), and i cant find a reference to what the “auto select branch” option should be. looking at the yaml from an existing pipeline shows only the same 3 options im trying to provide in the spec template (location, repo, path)
this might help: https://github.com/cloudposse/codefresh
Running out of resources @dustinvb
here are my current real blockers
Error response from daemon: Error processing tar file(exit status 1): write /GameClient/Assets/Scenes/CoralReefRewardScene/R&D/Scenes/RS-LayerTest.unity: no spac e left on device
@oriel FYI
Okay this is why I put you on enterprise to begin with. I knew you were going to be building in a way that you’d need extra space. I believe these pipelines are around 40GB. Do you have an estimate on how big these Docker images might get?
@francisco Could you assist here and just check this build out? We’re building using unity and the images are going to be very large. I am not sure but I think we’re pressing the storage limit defined in the runtime.
40GB seems a lot
Lemme check what’s our real size on Jenkins
Docker source image with UNity is 7GB
my full local repo that build 3 targets is around 8GB
now are you including the git files themselves ? .git ?
my .git looks like 20Gb
Hmm well that should have fit. Let me check with my support team on this.
General questions about best strategy to generate artifacts ala Jenkins
we used to zip and upload to S3. I did add the S3 integration.
Do you have any example on how to zip and upload ?
We deploy as well as a static website - still on S3 - but not zipped this time. any useful plugin ?
@dustinvb I am planning to use directly the aws s3-cli ? Is there anything simpler ? I do not see in the doc how to use the s3 artifact integration ? should I deal with the credentials in the script itself ?
I am looking for a step to aid with archive now.
For that s3 copy you just need to provide a service account in S3 with bucket access using these variables in the pipeline variables.
AWS_ACCESS_KEY_ID=<id> AWS_SECRET_ACCESS_KEY=<key> AWS_DEFAULT_REGION=<region>
If you plan on using this user over and over you may want to put these variables into a shared config you can import for other builds.
Sorry haven’t forgot still looking. Steps that often require this have this build in. Trying to find an a-la-carte example.
ArchiveFiles:
image: alpine:latest
commands:
- apk add --update zip
- tar -czvf ./archive.tgz ./folder/or/file.txt
I do not see that we have a step for this.
Please try the step below using an alpine image. I am checking with team about OOTB step.
perfect thanks !
ive traced my issue down to codefresh create -f failing no matter what i give it, giving me Error: "{\"status\":500,\"code\":\"1001\",\"name\":\"INTERNAL_SERVER_ERROR\",\"message\":\"Internal server error\",\"context\":{}}" not really sure where to go from here
You go right here… Sorry about the delay. I am traveling and often offline on weekends.
Please send me your YAML file for review. It is possible you’re missing a value or somethign.
I have some recent examples I updated here for comparison.
https://github.com/codefresh-contrib/example-voting-app/tree/master/.codefresh/specs
thanks for the reply! https://gist.github.com/mgeist/163026ccfb89626808ec24ca647dc589 using https://codefresh-io.github.io/cli/pipelines/spec/ as a reference, specifically “Pipeline with a remote spec template brought from a git repository”
Can you please try adding a name to your git context and let me know if that corrects the behavior? https://github.com/codefresh-contrib/example-voting-app/blob/master/.codefresh/specs/cd.yaml#L13
Name can be anything to identify the trigger.
I will make sure the team updates our spec examples if this resolves the issue.
same thing, code 1001, 500 internal server error
Can you update your example yaml with the latest?
Do you have a GIT context configured? Can you add these two fields?
context being the name of your context. If not, at a minimum please try to add the default revision.
# codefresh get contexts
NAME TYPE
CF_HELM_DEFAULT helm-repository
github git.github
and updated script: https://gist.github.com/mgeist/163026ccfb89626808ec24ca647dc589 no dice
finally got to the bottom of this, turns out pipeline names will break if they contain /. thanks for the assistance and FYI @dustinvb
Didn’t expect that to be the case. I will discuss with the team exposing this as an error on create.
It seems like a step in my codefresh.yml ends and proceeds to the next step before the final command exits
here’s what I’m trying to do:
pull down terraform and unzip the archive
here’s the steps in my yaml file:
steps:
clone_repo:
stage: prepare
type: git-clone
repo: '${{CF_REPO_OWNER}}/${{CF_REPO_NAME}}'
revision: '${{CF_REVISION}}'
git: github
wtf:
stage: prepare
image: alpine:latest
commands:
- 'apk add curl wget unzip file'
- 'ls -l'
- 'curl <https://releases.hashicorp.com/terraform/${{TERRAFORM_VERSION}}/terraform_${{TERRAFORM_VERSION}}_linux_amd64.zip> -o /tmp/terraform.zip'
next_part:
stage: prepare
title: 'hello world'
image: alpine:latest
commands:
- 'ls -lh'
- 'unzip /tmp/terraform.zip'
here’s some things I’ve tried:
- fixing some nice typos
- changing the step where
curlruns - running
curlwith-s?
a common error that I get that makes me think the step is ending before the curl exits is this:
End-of-central-directory signature not found. Either this file is not
a zipfile, or it constitutes one disk of a multi-part archive.
thrown by unzip
hey at least it’s friday tho ¯_(ツ)_/¯
my work around for this was just to put these into a dockerfile and build it. that works really well
I am glad you found a work around. Please note that while we do participate here our Intercom “chat” bubble is staffed 7am-6pm PDT M-F whenever these issues come up please check with the support staff there. We also have hours outside of the ones mentioned.
thanks @dustinvb I’ll use the chat bubble next time
2019-05-20
Hi, I’m trying to get a list of files that changed from current branch off a different branch. e.g. git diff $BRANCH –name-only on a pipeline. Seems that that other branches don’t get fetched so I ran a git fetch $BRANCH, but git asks for credentials. Got ssh key in place as suggested on the documentation but it’s not getting picked up by git. Any ideas on what I’m missing or a workaround for this?
client_ui: title: Building Game UI image: ${{client_image}} stage: build_client working_directory: ${{main_clone}} commands: - mkdir -p ~/.ssh - ssh-keyscan -H github.com >> ~/.ssh/known_hosts - echo “${SSH_KEY}” | tr '”${SPLIT_CHAR}”' ‘\n’ > ~/.ssh/id_rsa - chmod 600 ~/.ssh/id_rsa - ‘/bin/bash docker-build-ui.sh’
thanks Olivier!!!
Cache wiped out on a yml change. is it expected ?
Looking a 2 successive build - 27 min with cache and next one end up to be 2 hours
@dustinvb is it expected ?
literally the only changes on the depot were the yml file of codefresh..
@Olivier Vanden Bos will check and get back to you as soon as possible. This is the same issue you described in the email you sent me right?
They all seem related. It is almost if the codefrsh/volume was shared accross pipelines and concurrent build could collide somehow
I managed to repro locally though - with simulating the volume persistent accross pipelines. I am still digging.
Found the issue, was on our side
2019-05-21
2019-05-22
GitHub Incident: delay in notification deliveries May 22, 17:18 UTC Identified - GitHub is experiencing an incident related to their Notifications. This impacts email and webhook deliveries.
More information here: https://www.githubstatus.com/incidents/nf4gcjtcsqdb
Public/Free Office Hours with Cloud Posse starting now!!
GitHub Incident: delay in notification deliveries May 22, 23:46 UTC Resolved - This incident has been resolved. May 22, 17:18 UTC Identified - GitHub is experiencing an incident related to their Notifications. This impacts email and webhook deliveries.
More information here: https://www.githubstatus.com/incidents/nf4gcjtcsqdb
2019-05-23
2019-05-28
2019-05-29
public #office-hours starting now! join us here: https://zoom.us/j/684901853
2019-05-30
what is happening in “Initializing Process”
just started using codefresh and im getting “TypeError: Cannot read property ‘branch’ of undefined”
adding a tag key to the build step fixed that error for me
can you share your yaml as a snippet?
version: '1.0'
stages:
- prepare
- build
steps:
main_clone:
title: Cloning repo
type: git-clone
repo: '${{CF_REPO_OWNER}}/${{CF_REPO_NAME}}'
revision: '${{CF_REVISION}}'
stage: prepare
git: github
build_docker_image:
title: Building Docker Image
type: build
stage: build
image_name: bug-tracker
working_directory: ./
dockerfile: Dockerfile
haha
adding tag to build_docker_image
aha! yes
do we get docker layer caching automatically w/codefresh
yes
the ui is so nice 
example of layer caching working
i see a borken image
in slack?
wierd
yeah slack does that sometimes.. refreshing or clicking will show it
if you click on it, it shows
i see it now
do you guys use the codefresh image registry too?
yep
we use that and ECR
for simpler setups without the requirements of ECR use the codefresh registry
ECR is trickier plus max 1000 tags per repo
(which means you cannot use same repo for prod and ci)
i see
bout to test out this layer caching
circle charges for it and its super fickle (barely ever works). but i believe codefresh has a persistent volume per account
PV per pipeline
i believe
@dustinvb
amazing
here are our pipelines to get you started
how come your build_image doesnt require tag
i think this is a recent thing
none of our pipelines have it, but they started breaking
ah
Yes, very recent. Already contacted my team about this to update in our docs.
And Yes PV per pipeline. When successful volume stored for next pipeline execution./
are we supposed to setup the webhook in gh ourselves or does codefresh do that automatically for us?
i see that, but no webhook was created in my repo
i just created it myself (not sure if the above is a bug or im understanding it incorrectly)
This should be happening automatically. Can you send me the time you setup the GIT Trigger, Trigger Name and the Pipeline you created this trigger on? I am also assuming you’re using GH SaaS?
what do you mean GH Saas?
yeah
So i had a git push that didnt trigger at this time Thu May 30 13:07:03 2019 -0700
thats PST btw
You said the webhook was not under the repository settings?
You had to add manually?
dming you privately
is there a way to resync your github repositories on codefresh? I created a new repo for shared codefresh configs (build, deploy, etc) and I’m trying to select that repo in the dropdown but it doesn’t show up
This should be syncing automatically but we could be seeing some cache at work here. Can you gather your OS and Browser and send that over to our support team using the Intercom Chat Bubble widget?
2019-05-31
is there a way to delete/update helm repos in the codefresh ui?