#github-actions (2023-04)

this is working ok, but what I do to seeting two target webpage is copied to /var/www/html and the app is copied to another destination example var/www

you can either comb through the action code to see if it supports that or you can just run the action twice, not the most efficient but it should work

ok when i solve it i’ll let you know

the issue was solve

nice, are you able to share the solution?

yes, the code is here

name: Deploy

on: [push]

jobs: deploy:

runs-on: ubuntu-latest

steps:
- uses: actions/checkout@master
- name: Copiar el contenido del repositorio con scp
  uses: appleboy/scp-action@master
  env:
    HOST: ${{ secrets.HOST }}
    USERNAME: ${{ secrets.USERNAME }}
    PORT: ${{ secrets.PORT }}
    KEY: ${{ secrets.SSHKEY }}
  with:
    source: "pagweb/*"
    target: "/var/www/html"
    -uses: actions/checkout@master
- name: Copiar el contenido de folder app con scp
  uses: appleboy/scp-action@master
  env:
    HOST: ${{ secrets.HOST }}
    USERNAME: ${{ secrets.USERNAME }}
    PORT: ${{ secrets.PORT }}
    KEY: ${{ secrets.SSHKEY }}
  with:
    source: "app/*"
    target: "/home/ubuntu/app"
    strip_components: 1

gotta get used to using code blocks

wrap :point_up: in three backticks `

/snippet is a good option too as that has type formatting

it’s just a little easier to read

snippets are annoying IMO because I don’t like the collapse/expand but I can see how they’d be helpful with really long posts

currently using Datadog, previously had a home grown solution that used a AWS Lambda (Golang) that was the handler of GitHub Webhooks that would record metrics to CloudWatch. It worked just fine but didn’t have any bells or whistles and was yet another thing that we “owned” so using a managed product is easier so worth the capex.

IMO meercode’s pricing looks silly. I essentially compare pricing against GitHub, meaning that your product should provide as much value for the same pricing. GitHub’s market price is $4/user and $21/user, so does meercode add 7x and ~3x the value of GitHub for their respective tiers

even if you go with GitHub’s original team pricing at $9 we’re still talking 3x

Datadog is $8/month per committer

and no it doesn’t provide 2x the value of GitHub, but it’s still worth it to us

Thanks @Darren Cunningham, very useful input. I am aware of the Datadog integration. Datadog collector is opensource so I thought maybe I can find inspiration there to build something to ship to prometheus. Then I found out there’s a prometheus exporter on github but it only ships one metric. Going with either would still mean owning it. We already pay for a Datadog alternative that doesn’t have this integration, so moving to Datadog is not an option.

so basically the same as the CP GHA example, but the first 2 steps are new (setup-go and go install ....)

Awesome! I did end up doing a whole lot of testing and I do have it all working now, but there are some odd things. The first big fix I found wasx just throwing quotes around things, like this:

        outputs: |
          url: "${{ env.URL }}"

which seems to fix the - issue because that character in yaml is technically for an array

Another odd thing I found is that an individual key doesn’t like to be given multiline yaml. For example if I have this input in my github action

  paths:
    description: 'A multiline yaml string of key/value pairs for each Name and Path to include in the comment'
    required: false
    default: |
      Application: application
      Dashboard: /dashboard

And then try to directly use it to the matrix write action like

          paths: ${{ inputs.paths }}

It ends up breaking because of the newlines and a similar map issue. I got around this currently by turning my multiline yaml into a singleline yaml with newline characters, passing that into the write, and then turning it back into a multiline yaml on the other side. for example

    - name: Process paths for output
      shell: bash
      run: |
        echo "paths=$(echo '${{ inputs.paths }}' | awk '{printf "%s\n", $0}')" >> $GITHUB_ENV

    - uses: cloudposse/[email protected]
      id: out
      with:
        matrix-step-name: ${{ github.job }}
        matrix-key: ${{ env.APPLICATION_NAME }}
        outputs: |
          url: "${{ env.URL }}"
          paths: '${{ env.paths }}'

And then on the read side doing a echo -e so that the newlines get expanded.

Happy to share more details with you if you’d like! Passing around the multiline yaml stuff gets annoying complicated in github actions I’ve found so this might be some edge cases

@hatz I create a PR with new test for your case

https://github.com/cloudposse/github-action-matrix-outputs-write/pull/17

Could you check that I get the issue right ?

Hey Igor! I think that looks right, specifically this line matches what I originally described: https://github.com/cloudposse/github-action-matrix-outputs-write/pull/17/files#diff-200e312521c2e2c013035cc73d18[…]b7a2dc4faaf802b71d0137a07c3f5dR30

Also I’m not sure how much it matters but I am currently using the yaml : | instead of the : |- - not sure if the newline at the end breaks anything or not but thought I’d mention it! example:

        outputs: |
          url: '${{ env.URL }}'
          paths: '${{ env.paths }}'

@hatz I’m rewriting the action to nodejs

otherwise it would be always edge cases that breaks everything

That is awesome! I’m currently building some internal composite actions for our engineers to do ephemeral deployments, and doing stuff straight in bash has been kind of a headache because of newlines and special characters and all that.

Shell gets brittle really fast when trying to do multiline stuff I’ve found. And github actions seems to behave slightly different than my own shell so it’s been a lot of experimenting

If you want any help or need someone to do any kind of testing let me know! I’ve got my side working (with some tweaks to pre/post-process) it with 0.3.1 and definitely happy to test new stuff!

@hatz https://github.com/cloudposse/github-action-matrix-outputs-write/tree/0.4.0

I cut the new release

Could you pls test your special chars issues?

for multiline unfortunately you have to use such workaround

this is the limitation of GitHub actions substitution.

Hey! I haven’t had a chance to test this out yet, been OOO for a few days. I will probably get this tomorrow or friday!

Hey @Igor Rodionov - My first initial test led me to believe this didn’t work but I’m trying a couple other things. It downloaded all 9 of the files that I expected to see, but the ‘output’ only had one entry for each object. Possible I might have something on my side so I’ll let you know!

Alright one error I’m finding so far is what it considers to be valid yaml.

paths: 'Application:
'
fd_name: 'frontend-asset-manager'
---------------------
YAMLParseError: Missing closing 'quote at line 2, column 21:

paths: 'Application:
                    ^

This seems to be related to the multi-line yaml that I have where the value is an empty string

here is my input

        outputs: |
          url: ${{ env.URL }}
          paths: '${{ inputs.paths }}'
          fd_name: ${{ env.FD_NAME }}

I have the closing quote on the second line but it doesn’t seem to see that

The new error output though is much better

Here’s the error I get if I don’t throw single quotes around it

Error: Outputs should be valid YAML 
---------------------
url: <https://frontend-usdf-382.figureapps.dev>
paths: Application:

fd_name: frontend-usdf
---------------------
YAMLParseError: Nested mappings are not allowed in compact mappings at line 2, column 8:

paths: Application:
       ^

I’ve tried with both outputs: | and outputs: |- too

try

Alright this worked on the write step now:

paths: '${{ toJson(inputs.paths) }}'

it ends up looking like this:

  with:
    matrix-step-name: ephemeral-deploy
    matrix-key: frontend-auth-382
    outputs: url: <https://frontend-auth-382.url.com>
      paths: '"Application:\n"'
      fd_name: frontend-auth

However, on the read side I’m still only actually seeing one entry.

No artifact name specified, downloading all artifacts
Creating an extra directory for each artifact that is being downloaded
starting download of artifact 1ad31b158455284757e5320e8e4ca65ed4c909fce6554d4713a764e6480d9f11 : 1/9
Total number of files that will be downloaded: 1
starting download of artifact 1de3311f3a5fb2f26beb247ba1ee2e3866eac7f2cd014ca2b940117f87d46d22 : 2/9
Total number of files that will be downloaded: 1
starting download of artifact 4fc8f69a7cd20390d89bc18ea6400ec381bfea1ff6fe77c64d0b0439ab3de809 : 3/9
Total number of files that will be downloaded: 1
starting download of artifact 4fea876fd1027053d0ddf38e74b805fbe8a4e1944613e2c7aecb501d79f1d512 : 4/9
Total number of files that will be downloaded: 1
starting download of artifact 6c4186c01f171e8d190e6b8688645a0591030d0cced58c86b353d565a0211bda : 5/9
Total number of files that will be downloaded: 1
starting download of artifact 86464d497618037bd50d8e9925b69000e8e433dae9c05a2a99e356fd93147d90 : 6/9
Total number of files that will be downloaded: 1
starting download of artifact a23bd2a5d63a624994e3258b3dc50af5a26bdb6fe7ec483b51a89546f99ba47b : 7/9
Total number of files that will be downloaded: 1
starting download of artifact bf4c9fa23d050fb0912454d8a2a9d6d40f16c348289320fde9e4d9e59f21d635 : 8/9
Total number of files that will be downloaded: 1
starting download of artifact d7a846ded11eb21020889310d301d57d579499d0034000632dd9dfe1f27571c8 : 9/9
Total number of files that will be downloaded: 1
There were 9 artifacts downloaded

Everything gets downloaded. but then the json seems to not be merging them together

        json=$(echo '${{ steps.read-matrix.outputs.result }}')
        echo $json

{"url":{"matrix_key":"<https://frontend.url.com>"},"paths":{"matrix_key":"\"Application:\\n\""},"fd_name":{"matrix_key":"frontend"}}

also worth noting that no single quotes had a slightly different write input

paths: ${{ toJson(inputs.paths) }}

became

  paths: "Application:\n"

but the read was the same issue

Manually inspecting all 9 of the artifacts that got uploaded during the job and I see what I expect for each one, so it seems the merging on the read somehow isn’t quite working

example artifact uploads (all 9 are different values but same keys)

{"matrix_key":{"url":"<https://frontend-auth.url.com>","paths":"Application:\n","fd_name":"frontend-auth"}}

{"matrix_key":{"url":"<https://frontend-group.url.com>","paths":"Application:\n","fd_name":"frontend-group"}}

Oh I see the issue. It seems like that json structure changed. I think the current version of the -write action is now incompatible with the -read action. Here’s an example json from 0.3.1 release:

{"frontend-auth":{"url":"<https://frontend-auth.url.com>","paths":"Application:\\n\\n","fd_name":"frontend-auth"}}

I think the bug is that it is using the matrix_key as a string literal instead of a value ref

@Igor Rodionov Alright I submitted a PR for that specific issue here: https://github.com/cloudposse/github-action-matrix-outputs-write/pull/20

I’m now running into a new issue where the newline character in the json is not being evaluated properly (because in github actions I think it wants \\n instead of \n, but at least the first issue should be fixed)

Alright I fixed my issue around the newline character, in my post-processing of the read action I had some weird stuff to handle the \\n and I forgot to remove it in my testing, so it’s all working well with the above PR fix I made + the toJson() you recommended!

@Igor Rodionov can you check that PR out today? I’d love to get it merged/released so we can start using it internally!

Also can you elaborate on why you do an assertion for empty fields? I have a case where passing empty fields is valid, seems like a bit of an over-reaching assertion on the input data to not allow empty string. Thoughts?

@Igor Rodionov

  with:
    matrix-step-name: ephemeral-deploy
    outputs: url: ''
  paths: 'Application:\n\n'
  fd_name: ''
  errors: '\n\n

\n:x: argocd_token is null. Please reach out to #devops so we can help you onboard.\n

'

e.g. if an error happens I’m passing errors through, but that means other fields (like my url) never got created so that becomes ''

Error: Expected '' to not equal ''

I Created the bucket s3 I only see arn ejemplo: arns3:::test-github-action02 but the url service no

The question is I should do a procedure to create it

if anyone has a step by step, I would appreciate it

I found this documentation

this command runer in console of my github Enterprise server ?

yes now I get it the finger

Awesome!! Thanks for the release