#variant (2019-03)

https://github.com/mumoshu/variant

Discuss variant (the “Universal CLI”) https://github.com/mumoshu/variant

Archive: https://archive.sweetops.com/variant/

2019-03-30

mumoshu avatar
mumoshu

alright. will address it soon. thx!

2019-03-29

mumoshu avatar
mumoshu

@oscarsullivan_old Try running any of the defined tasks by specifying the name like ./myfirstcommand foo!

I guess - you’d expected to see the command’s help when it was run without any args, right? I think it’s currently failing, by design, because you have no root with script defined. I can make variant just print the help instead, though.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@mumoshu I think printing help would make sense in this case

2019-03-28

oscarsullivan_old avatar
oscarsullivan_old

Any clue why I get this error when following the tutorial?

 ⧉  sandbox 
 ✓   (-sandbox-admin) variant ⨠ cat myfirstcommand 

\#!/usr/bin/variant

tasks:
  bar:
    script: |
      echo "dude"
  foo:
    parameters:
    - name: bar
      type: string
      description: "the bar"
    - name: environment
      type: string
      default: "heaven"
    script: |
      echo "Hello {{ get "bar" }} you are in the {{ get "environment" }}"
 ⧉  sandbox 
 ✓   (-sandbox-admin) variant ⨠ which variant 
/usr/bin/variant
 ⧉  sandbox 
 ✓   (-sandbox-admin) variant ⨠ ./myfirstcommand
Error: unknown command "./myfirstcommand" for "myfirstcommand"
Run 'myfirstcommand --help' for usage.
Unexpected type of error *errors.errorString: unknown command "./myfirstcommand" for "myfirstcommand"
ls -la
-rwxrwxr-x  1 oscar oscar  308 Mar 28 13:21 myfirstcommand
antonbabenko avatar
antonbabenko

chmod +x ./myfirstcommand ?

:--1:2
oscarsullivan_old avatar
oscarsullivan_old

oops I missed the ls -la, but yes that’s already been run

oscarsullivan_old avatar
oscarsullivan_old
-rwxrwxr-x  1 oscar oscar  308 Mar 28 13:21 myfirstcommand
Joe Presley avatar
Joe Presley
05:22:09 PM

@Joe Presley has joined the channel

2019-03-27

ldlework avatar
ldlework
09:25:06 PM

@ldlework has joined the channel

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
09:26:51 PM

@Erik Osterman (Cloud Posse) set the channel topic: https://github.com/mumoshu/variant

loren avatar
loren
09:28:48 PM

@loren has joined the channel

oscarsullivan_old avatar
oscarsullivan_old
11:59:47 PM

@oscarsullivan_old has joined the channel

2019-03-26

2019-03-25

dennybaa avatar
dennybaa
09:43:53 AM

Hello guys, I wonder is it possible to pass the parent task parameter as an argument to the child?

dennybaa avatar
dennybaa

oh shh, sorry it worked when quoting opt1: '{{ get "test_param" }}'. @mumoshu great tool btw, thank you.

:1000:3
1
dennybaa avatar
dennybaa

I’ve been playing with variant and in my case I feel like I’m missing toObject template helper. Does anyone now how to treat the output of a task as an object? Is it possible in variant?

mumoshu avatar
mumoshu

@dennybaa hey! you can define an parameter as type: object so that any JSON-formatted stdout content can be parsed as object`

mumoshu avatar
mumoshu
$ cat Variantfile

\#!/usr/bin/env variant

tasks:
  getjson:
    script: |
      echo '{"key1":1, "key2":2}'
  test:
    parameters:
    - name: getjson
      type: object
      properties:
        key1:
          type: integer
        key2:
          type: integer
    script: |
      cat <<EOS
      {{ get "getjson" | toYaml }}
      EOS
$ variant test
variant ≫ starting task getjson
variant.getjson ≫ {"key1":1, "key2":2}
variant ≫ starting task test
key1: 1
key2: 2
mumoshu avatar
mumoshu

btw, you can even source from/override the parameter value with a file:

$ cat my.json
{"foo":11,"bar":22}
$ variant test --getjson=my.json
variant ≫ starting task test
bar: 22
foo: 11
dennybaa avatar
dennybaa

@mumoshu yup, I’ve bumped into it as well. However there is a correlating issue - “get” tasks can not have parameterized arguments. Correct me if I’m mistaken, please.

dennybaa avatar
dennybaa
tasks:
  key:
    parameters:
    - name: count
      type: integer
      default: 1
    script: |
      echo '["private", "public"]'

  create:
    options:
    - name: key
      required: true
    script: |
      echo {{ get "key" | toYaml }}
mumoshu avatar
mumoshu

@dennybaa you’d need to add type: object for create, and return a JSON object, not JSON array from the key task

dennybaa avatar
dennybaa

In fact I already use pattern as above. So I guess parameters should work, sorry. I see. Yaml is not possible with type: object ?

mumoshu avatar
mumoshu

the point would be that you don’t need to explicitly parse json or yaml. type: object do that for you!

mumoshu avatar
mumoshu

would you mind sharing several example variantfiles you’ve tried, so that i can better get what you’re trying and rewrite them for you?

dennybaa avatar
dennybaa

Thank you very much ! It’s just something wasn’t working, but now after a day of trail with variant I feel much more confident. So I will try to brush the code up first and get back to you after and for refactoring advises.

mumoshu avatar
mumoshu

great!

mumoshu avatar
mumoshu

@dennybaa i was in the impression that you wanted to (1) internally generate an parameterized array and (2) print the result as yaml

mumoshu avatar
mumoshu
$ cat Variantfile2

\#!/usr/bin/env variant

tasks:
  myarray:
    parameters:
    - name: second
      type: string
      default: "public"
    script: |
      echo '["private", "{{get "second"}}"]'

  print-json-array-as-yaml:
    options:
    - name: myarray
      type: array
    script: |
      echo '{{ get "myarray" | toYaml }}'
$ variant Variantfile2 print-json-array-as-yaml
Variantfile2 ≫ starting task myarray
Variantfile2.myarray ≫ ["private", "public"]
Variantfile2 ≫ starting task print-json-array-as-yaml
- private
- public

dennybaa avatar
dennybaa

@mumoshu why )?

mumoshu avatar
mumoshu

(cont) in that case the above may be of help…

mumoshu avatar
mumoshu

or for type: object, i’d write something like:

$ cat Variantfile2

\#!/usr/bin/env variant

tasks:
  myobject:
    parameters:
    - name: second
      type: string
      default: "public"
    script: |
      echo '{"items":["private", "{{get "second"}}"]}'

  print-json-object-as-yaml:
    options:
    - name: myobject
      type: object
    script: |
      echo '{{ get "myobject" | toYaml }}'

which produces:

$ variant Variantfile2 print-json-object-as-yaml
Variantfile2 ≫ starting task myobject
Variantfile2.myobject ≫ {"items":["private", "public"]}
Variantfile2 ≫ starting task print-json-object-as-yaml
items:
- private
- public
dennybaa avatar
dennybaa

I was just trying to say that now I realize that parameterization actually works as well as object. So I should give them a better touch)

dennybaa avatar
dennybaa

@mumoshu here it’s what I’m trying to do

tasks:
  key:
    parameters:
    - name: count
      type: integer
      default: 1
    script: |
      echo '0: ["private", "public"]'

  create:
    options:
    - name: key
      type: object
      required: true
    script: |
      echo '{{ get "key" | toYaml }}'
mumoshu avatar
mumoshu

what do you want to use count for?

dennybaa avatar
dennybaa
tasks:
  key:
    description: Generates eos key-pair(s)
    parameters:
    - name: count
      description: Number of key-pairs to create
      type: integer
      default: 1
    runner:
      image: eosio/eos
      command: bash
      args: [-c]
    script: |
      {{- range $i, $e := until (.count) }}
      pair=$(cleos create key --to-console | sed 's/.*: //' | sed ':a;N;$!ba;s/\n/", "/g')
      printf "{{ $i }}: [\"%s\"]\n" "$pair"
      {{- end }}

  create:
    description: Creates a keypair yaml file

    options:
    - name: key
      type: object
      required: true
    script: |
      cat <<EOF
      {{ get "key" | toYaml }}
      EOF
dennybaa avatar
dennybaa

This is the real one ^

dennybaa avatar
dennybaa

first one is a simplification

mumoshu avatar
mumoshu

seems like you want to internally generate a YAML data and print it, right?

dennybaa avatar
dennybaa

yes so far yes. But I’m going to process that data

mumoshu avatar
mumoshu

ah

dennybaa avatar
dennybaa

so the data is not sucked by create…

mumoshu avatar
mumoshu

then i have to go back to yout original question.. you can’t map yaml string to type object

mumoshu avatar
mumoshu

you need to use json as the internal data format

mumoshu avatar
mumoshu

later print it as yaml by pipeing it to toYaml

mumoshu avatar
mumoshu

toYaml is for marshalling objects to YAML, not for parsing YAML data into objects

dennybaa avatar
dennybaa

i understand

:--1:1
dennybaa avatar
dennybaa

that I was saying yaml can not be passed as type: obeject

dennybaa avatar
dennybaa

the point would be that you don’t need to explicitly parse json or yaml. type: object do that for you!

dennybaa avatar
dennybaa

?

dennybaa avatar
dennybaa

can we make it possible too?

dennybaa avatar
dennybaa

Imagine I want to load file a.yml

dennybaa avatar
dennybaa

But first we need to generate that data too… (

dennybaa avatar
dennybaa

while this kind of stuff works

tasks:
  key:
    parameters:
    - name: count
      type: integer
      default: 1
    script: |
      echo '{"items": ["private", "public"]}'

  create:
    options:
    - name: key
      type: object
      required: true
    script: |
      echo '{{ index (get "key") "items" }}'
mumoshu avatar
mumoshu

that I was saying yaml can not be passed as type: obeject

mumoshu avatar
mumoshu

can we make it possible too?

1
dennybaa avatar
dennybaa

Would be awesome! Meanwhile I’m sticking to json

:--1:1
mumoshu avatar
mumoshu

@dennybaa is it possible for you to use jq, so that you can convert it to json before outputting?

https://github.com/mikefarah/yq

mikefarah/yq

yq is a portable command-line YAML processor. Contribute to mikefarah/yq development by creating an account on GitHub.

mumoshu avatar
mumoshu
    script: |
      bash <<EOC | yq r -j -
      {{- range $i, $e := until (.count) }}
      pair=$(cleos create key --to-console | sed 's/.*: //' | sed ':a;N;$!ba;s/\n/", "/g')
      printf "{{ $i }}: [\"%s\"]\n" "$pair"
      {{- end }}
      EOC
dennybaa avatar
dennybaa

@mumoshu yup, i’m aware of yq bro of jq. Surely it’s possible, though it’s not likely to use option in general.

dennybaa avatar
dennybaa

If we refer to containers, it will be a no go to inject yet another binary into them…

mumoshu avatar
mumoshu

i slightly agree with u i used to think similarly, but started giving up because i can’t complete everything with just variant(of course?

dennybaa avatar
dennybaa

In my case personal case I have a customized container… i can install yq, jq In other cases I’m pretty sure possibility to feed yaml as object should exist. This will allow users to build interesting tools

mumoshu avatar
mumoshu

agreed

mumoshu avatar
mumoshu

it isn’t actually a bad idea to add yaml->object conversion for variant params

mumoshu avatar
mumoshu

as it doesn’t add additional binary size to variant…

dennybaa avatar
dennybaa


but started giving up because i can’t complete everything with just variant(of course?
I believe the grade of your integration issues is above one I can now imagine

1
mumoshu avatar
mumoshu
tasks:
  key:
    description: Generates eos key-pair(s)
    parameters:
    - name: count
      description: Number of key-pairs to create
      type: integer
      default: 1
    output:
      format: yaml
    script: |
      {{- range $i, $e := until (.count) }}
      pair=$(cleos create key --to-console | sed 's/.*: //' | sed ':a;N;$!ba;s/\n/", "/g')
      printf "{{ $i }}: [\"%s\"]\n" "$pair"
      {{- end }}

maybe it’s a matter of adding output.format to the task definition?

dennybaa avatar
dennybaa

i don’t think it’s needed… better just raw understanding of json or yaml as object, since we don not introduce any syntax or functions (toObject, fromYaml, fromJson )…

dennybaa avatar
dennybaa

anyways. json is a subset of yaml, so I bet just one line of code should be changed somewhere where this conversion happens)

mumoshu avatar
mumoshu

aha, good point. i was implicitly forced to consider other formats like toml, jsonnet, that might be feature-requested in the future

mumoshu avatar
mumoshu

@dennybaa variant v0.27.0 has been published with this feature: type: object parameter sourced from a task output in YAML

dennybaa avatar
dennybaa

wow. amazing !

2019-03-22

2019-03-21

mumoshu avatar
mumoshu

Yeah both use-cases make sense to me! From the author’s perspective, I’d say Variantfile is an alternative to putting Makefile in your project root. It isn’t intended to be reused by any other projects

tolstikov avatar
tolstikov

exactly, that’s the one of my cases! So it is good to have options here!

:--1:1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Makes sense….

mumoshu avatar
mumoshu

./yourcmd way, like kopsctl, should be used whenever it is being reused in any way, like (1) imported from another variant command via import: or (2) as a single executable binary

mumoshu avatar
mumoshu

Btw, for the latter, you can go build your variant command like explained in https://github.com/mumoshu/variant#releasing-a-variant-made-command, or in a bit more sophisticated way https://github.com/mumoshu/variant/blob/master/examples/hello/hack/generate-maingo-struct#L1-L15.

mumoshu/variant

Wrap up your bash scripts into a modern CLI today. Graduate to a full-blown golang app tomorrow. - mumoshu/variant

mumoshu/variant

Wrap up your bash scripts into a modern CLI today. Graduate to a full-blown golang app tomorrow. - mumoshu/variant

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
genuinetools/binctr

Fully static, unprivileged, self-contained, containers as executable binaries. - genuinetools/binctr

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

This plus variant!

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

I have thought about how we could do this with geodesic

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

So you can just run it with out even Docker installed

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Remember the problem you we’re trying to solve about installing all the deps of variant

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

In this model, you would have a dockerfile of deps, a variant entry point, and a compiled container exe

mumoshu avatar
mumoshu

that sounds very interesting

mumoshu avatar
mumoshu

fyi: if you don’t want the dependency to the bash installed on your machine: https://github.com/progrium/go-basher

progrium/go-basher

Library for writing hybrid Go and Bash programs. Contribute to progrium/go-basher development by creating an account on GitHub.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Dude! That’s awesome

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

I think that’s a step in the right direction

2019-03-20

tolstikov avatar
tolstikov

is there any specific reason to separate config file & main Variantfile or config could be included in Variantfile file (as an option)?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@tolstikov you’re not using the #! mode?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
cloudposse/geodesic

Geodesic is a cloud automation shell. It&#39;s the fastest way to get up and running with a rock solid, production grade cloud platform built on top of strictly Open Source tools. ★ this repo! h…

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

to me this is the ideal invocation

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

then the . configs are quite nice

tolstikov avatar
tolstikov

@Erik Osterman (Cloud Posse) probably my question was not clear

I’ve meant that I like the idea to keep everything related to Variant in a single file (including environment specific config files, a main config file, etc) for some specific projects, where some other people will participate (it can be Variantfile or !# file, etc).

It will be probably easier for these people to adopt to the new workflow (with Variant) when everything is in the one place (file).

Also, it will reduce “invasion” (only one new file instead of 2-3 files) to other people’s projects (where I only setup the CICD for example).

Please let me know if I’m not clear!

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Personally, the Variantfile approach was less appealing than writing a cli with variant

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

for example, we write our cli called kopsctl using the variant DSL

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

and then as a cli tool, I want the configuration separate. that logical separation IMO makes it more manageable than keeping it all in the same script.

tolstikov avatar
tolstikov

yes, it makes total sense in this context

tolstikov avatar
tolstikov

but, I still can see some cases where Variantfile would be more appealing to some people

tolstikov avatar
tolstikov

as for me only, I prefer the CLI approach as well

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

ok,

tolstikov avatar
tolstikov

but still like to have some options

tolstikov avatar
tolstikov

thanks for sharing your vision!

Tim Malone avatar
Tim Malone
11:07:47 PM

@Tim Malone has joined the channel

2019-03-19

tolstikov avatar
tolstikov

regarding https://github.com/mumoshu/variant/issues/51

it is helpful to include single tasks, but if I want to include a bunch of tasks with a single statement, what are the options (I didn’t check all the tests & code)?

@mumoshu

Import variant commands from various versioned sources · Issue #51 · mumoshu/variant

So that you can create a common/library variant commands to be versioned and reused. tasks: anothercmd: # Basically runs go-getter <http://github.com/foo/bar?ref=v0.1.0//dir|github.com/foo/bar?ref=v0.1.0//dir> /tmp/somewhere and then inclu…

tolstikov avatar
tolstikov

found this one - https://github.com/mumoshu/variant/issues/52 seems like this is it

Import config files from various versioned sources · Issue #52 · mumoshu/variant
So that you can create a common config file that is versioned and reused in various variant commands in divergent places. tasks: mytask: parameters: - name: config type: object script: cat <&l…
tolstikov avatar
tolstikov

nope, that’s only for config files in fact

tolstikov avatar
tolstikov

just import in the main file works OK:

...
import: variant_lib.yaml
...

good stuff!

mumoshu avatar
mumoshu

awesome to see you figured out about that! yes, import can be used for importing both single task or a set of tasks :)

mumoshu avatar
mumoshu

also - if you encouter any difficulty on writing your variant command, pls feel free to share an example code!

:--1:1
tolstikov avatar
tolstikov

thanks! I believe this tool is going to save a lot of time for a lot of people!

I’m still going to figure out the differences between “inputs”, “options” & “parameters”

1
mumoshu avatar
mumoshu

those are interchangeable, and parameters is the latest and recommended syntax! the former two will be eventually removed

tolstikov avatar
tolstikov

I see, thanks for the explanation!

    keyboard_arrow_up