#random (2023-05)

Non-work banter and water cooler conversation

A place for non-work-related flimflam, faffing, hodge-podge or jibber-jabber you’d prefer to keep out of more focused work-related channels.

Archive: https://archive.sweetops.com/random/

2023-05-01

Sudhish KR avatar
Sudhish KR

Hi Folks, I am Sudhish - https://www.sudhishkr.com/ - and I am a Software Engineer by profession from San Francisco. I have worked extensively in the Platform/Infra/Data side in the last decade. Looking fwd to the learning.

On the side, I do a lot of consulting. Checkout mytytech.com - and we would be happy to help you out.

SudhishKR

Sudhish KR’s personal website

2023-05-04

Alex Atkinson avatar
Alex Atkinson

Wow. This Krakend (api gw) guy does an amazing job of showing off their kit. https://www.youtube.com/watch?v=VtXXZRO84t8

I feel like Kong used to have content a lot like this, but it’s hard to find now. Did they prune it to posture their enterprise offering more prominently? IDK. Anyway, Krakend looks sick. But what are folks around here using lately?

2023-05-07

daniel613 avatar
daniel613

Please reach out and/or check out our website, www.southlakeconsulting.com!

2023-05-08

Alex Atkinson avatar
Alex Atkinson

I’m trying to find some of the first/early references to a multi-environment product pipeline. Does anyone have a line on something like this? I found this interesting doc sponsored by the US Army Computer Systems Command and The IEEE Computer Society that shows:

The Software Life Cycle Management Modle (Fig. 1)

We’re still struggling with the same qa/ops concerns as ever… Mostly. (Figure 1. Factors in Software Quality) But it doesn’t explicitly state the definition of environment, or align it with “dev, qa, stag, prod…”.

Alex Atkinson avatar
Alex Atkinson

Here’s a paper from 1990 that starts to talk about the SDLC, and testing. Interesting read, if short. https://eli.johogo.com/pdf/jsm-90.pdf

… I’ll bread crum into the first paper I find that specifically defines what I’m looking for.

Alex Atkinson avatar
Alex Atkinson

The Handbook of Software Quality Assurance Techniques Applicable to the Nuclear Industry (1987) has the first graph showing cost to remediate an issue vs delivery phase that I know of on page 10.2. This paper reads a lot like most compliance frameworks… https://inis.iaea.org/collection/NCLCollectionStore/_Public/19/018/19018784.pdf

Alex Atkinson avatar
Alex Atkinson

HITACHI: PIONEERING A “FACTORY” STRATEGY AND STRUCTURE FOR LARGE-SCALE SOFTWARE DEVELOPMENT Page 18: “Around 1970 we came to believe that we had to introduce a components control system similar to what you find for hardware…” Page 21: Staffing allocation: 55% Planning & Design, 5% coding, 35% debugging, 10% inspection….. Page 32: The broke Brooks Law… Adding people to deliver on time worked? WHAT? https://dspace.mit.edu/bitstream/handle/1721.1/48057/hitachipioneerinx00cusu.pdf

Alex Atkinson avatar
Alex Atkinson

Then there’s a bunch of talk about mitigating “merge hell” with Git vs SVN… I remember that. And Agile comes along with DevOps. I went from Ops (Microsoft bs) to DevOps (linux yay) when devops became a thing… It seems I took the devops pipeline for granted, having started with this classic.

Leah Blank avatar
Leah Blank

hey y’all! Trying to sign up for your newsletter, but the form seems broken. Any ideas what’s up?

venkata.mutyala avatar
venkata.mutyala

@Erik Osterman (Cloud Posse)

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Hey @Leah Blank thanks for letting me know. I’ll check

2023-05-10

Alex Atkinson avatar
Alex Atkinson

Does anyone use opensource kong with open source openid connect baked in, instead of paying for Enterprise to get the feature? I found this repo that looks promising, but wonder what other folks are using for api gw these days. https://github.com/revomatico/docker-kong-oidc There’s Tyk, which is open source with all features, including oidc… https://github.com/TykTechnologies/tyk

tommy avatar
attachment image

title: openid-connect
keywords:

• Apache APISIX • API Gateway • OpenID Connect • OIDC
description: OpenID Connect allows the client to obtain user information from the identity providers, such as Keycloak, Ory Hydra, Okta, Auth0, etc. API Gateway APISIX supports to integrate with the above identity providers to protect your APIs.


Description

OpenID Connect (OIDC) is an authentication protocol based on the OAuth 2.0. It allows the client to obtain user information from the identity provider (IdP), e.g., Keycloak, Ory Hydra, Okta, Auth0, etc. API Gateway Apache APISIX supports to integrate with the above identity providers to protect your APIs.

Attributes

NOTE: encrypt_fields = {"client_secret"} is also defined in the schema, which means that the field will be stored encrypted in etcd. See encrypted storage fields.

Scenarios

:::tip

Tutorial: Use Keycloak with API Gateway to secure APIs

:::

This plugin offers two scenorios:

  1. Authentication between Services: Set bearer_only to true and configure the introspection_endpoint or public_key attribute. In this scenario, APISIX will reject requests without a token or invalid token in the request header.
  2. Authentication between Browser and Identity Providers: Set bearer_only to false. After successful authentication, this plugin can obtain and manage the token in the cookie, and subsequent requests will use the token. In this mode, the user session will be stored in the browser as a cookie and this data is encrypted, so you have to set a key for encryption via session.secret.

Token introspection

Token introspection validates a request by verifying the token with an OAuth 2.0 authorization server.

You should first create a trusted client in the identity server and generate a valid JWT token for introspection.

The image below shows an example token introspection flow via a Gateway:

token introspection

The example below shows how you can enable the Plugin on Route. The Route below will protect the Upstream by introspecting the token provided in the request header:

curl <http://127.0.0.1:9180/apisix/admin/routes/5> -H 'X-API-KEY: edd1c9f034335f136f87ad84b625c8f1' -X PUT -d '
{
  "uri": "/get",
  "plugins":{
    "openid-connect":{
      "client_id": "${CLIENT_ID}",
      "client_secret": "${CLIENT_SECRET}",
      "discovery": "${DISCOVERY_ENDPOINT}",
      "introspection_endpoint": "${INTROSPECTION_ENDPOINT}",
      "bearer_only": true,
      "realm": "master",
      "introspection_endpoint_auth_method": "client_secret_basic"
    }
  },
  "upstream":{
    "type": "roundrobin",
    "nodes":{
      "httpbin.org:443":1
    }
  }
}'

Now, to access the Route:

curl -i -X GET <http://127.0.0.1:9080/get> -H "Host: httpbin.org" -H "Authorization: Bearer {JWT_TOKEN}"

In this example, the Plugin enforces that the access token and the Userinfo object be set in the request headers.

When the OAuth 2.0 authorization server returns an expire time with the token, it is cached in APISIX until expiry. For more details, read:

  1. lua-resty-openidc’s documentation and source code.
  2. exp field in the RFC’s Introspection Response section.

Introspecting with public key

You can also provide the public key of the JWT token for verification. If you have provided a public key and a token introspection endpoint, the public key workflow will be executed instead of verification through an identity server. This is useful if you want to reduce network calls and speedup the process.

The example below shows how you can add public key introspection to a Route:

curl <http://127.0.0.1:9180/apisix/admin/routes/5> -H 'X-API-KEY: edd1c9f034335f136f87ad84b625c8f1' -X PUT -d '
{
  "uri": "/get",
  "plugins":{
    "openid-connect":{
      "client_id": "${CLIENT_ID}",
      "client_secret": "${CLIENT_SECRET}",
      "discovery": "${DISCOVERY_ENDPOINT}",
      "bearer_only": true,
      "realm": "master",
      "token_signing_alg_values_expected": "RS256",
      "public_key": "-----BEGIN PUBLIC KEY-----
      {public_key}
      -----END PUBLIC KEY-----"
    }
  },
  "upstream":{
    "type": "roundrobin",
    "nodes":{
      "httpbin.org:443":1
    }
  }
}'

In this example, the Plugin can only enforce that the access token should be set in the request headers.

Authentication through OIDC Relying Party flow

When an incoming request does not contain an access token in its header nor in an appropriate session cookie, the Plugin can act as an OIDC Relying Party and redirect to the authorization endpoint of the identity provider to go through the OIDC authorization code flow.

Once the user has authenticated with the identity provider, the Plugin will obtain and manage the access token and further interaction with the identity provider. The access token will be stored in a session cookie.

The example below adds the Plugin with this mode of operation to the Route:

curl <http://127.0.0.1:9180/apisix/admin/routes/5> -H 'X-API-KEY: edd1c9f034335f136f87ad84b625c8f1' -X PUT -d '
{
  "uri": "/get",
  "plugins": {
    "openid-connect": {
      "client_id": "${CLIENT_ID}",
      "client_secret": "${CLIENT_SECRET}",
      "discovery": "${DISCOVERY_ENDPOINT}",
      "bearer_only": false,
      "realm": "master"
    }
  },
  "upstream": {
    "type": "roundrobin",
    "nodes": {
      "httpbin.org:443": 1
    }
  }
}'

In this example, the Plugin can enforce that the access token, the ID token, and the UserInfo object to be set in the request headers.

Troubleshooting

  1. If APISIX cannot resolve/connect to the identity provider (e.g., Okta, Keycloak, Authing), check/modify the DNS settings in your configuration file (conf/config.yaml).
  2. If you encounter the error the error request to the redirect_uri path, but there's no session state found, please confirm whether the currently accessed URL carries code and state, and do not directly access redirect_uri.
  3. If you encounter the error the error request to the redirect_uri path, but there's no session state found, please check the redirect_uri attribute : APISIX will initiate an authentication request to the identity provider, after the authentication service completes the authentication and authorization logic, it will redirect to the address configured by redirect_uri (e.g., <http://127.0.0.1:9080/callback>) with ID Token and AccessToken, and then enter APISIX again and complete the function of token exchange in OIDC logic. The redirect_uri attribute needs to meet the following conditions:

redirect_uri needs to be captured by the route where the current APISIX is located. For example, the uri of the current route is /api/v1/*, redirect_uri can be filled in as /api/v1/callback; • scheme and host of redirect_uri (scheme:host) are the values required to access APISIX from the perspective of the identity provider.

1
tommy avatar
apache/apisix

The Cloud-Native API Gateway

tommy avatar

Or you can use traefik as well.

Yonatan Koren avatar
Yonatan Koren

If anyone wants to join a quick webinar I’m helping panel on secrets starting at 19:00 UTC (20m from now)

Not a product pitch — will be talking about k8s secrets operators, CI secrets, etc

https://hs.codefresh.io/merging-to-main-cicd-secrets

2

2023-05-12

Sherif Ayad avatar
Sherif Ayad

Hi all .. I have a question regarding a small yet important PR that I opened on Github for one of the repos and it’s kinda urgent .. can someone help with that and the review eventually?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Please post to #pr-reviews

1

2023-05-14

Yonatan Koren avatar
Yonatan Koren

How many full-time Platform Engineers / Cloud Engineers / DevOps Engineers are required for a cloud migration, where there are several production environments (k8s clusters, database and blob storage services, etc)

Say AWS to GCP

Just wondering what people would place this number at (random ballpark #)

Hao Wang avatar
Hao Wang

Normally will take 1-2 engineers and a part-time project manager

1
Yonatan Koren avatar
Yonatan Koren

I guess that would mean full time on that project and nothing else, right?

Hao Wang avatar
Hao Wang

oh yeah, may take 1 month to 3

bradym avatar

The number of engineers needed and how long it takes will depend entirely on your environment, how you are using AWS, how you plan to use GCP, how familiar your engineers are with both AWS and GCP (and the fact that this will likely vary significantly from one engineer to another), how directly services can be translated, how you’re running your applications, what gotchas are encountered along the way and many more factors.

Figuring it out should be a project itself.

1
Yonatan Koren avatar
Yonatan Koren


Figuring it out should be a project itself.
Well said

2023-05-15

2023-05-16

Alex Atkinson avatar
Alex Atkinson

Is whimsical good for cloud architecture diagrams? I can’t see anything on the site that says it would be, but…. ?

Yonatan Koren avatar
Yonatan Koren

If you can use it for wireframes you can use it for pretty much anything…

But hopefully there’s an option to keep all of your AWS service logos saved instead of having to copy and paste each time you make a new diagram

Soren Jensen avatar
Soren Jensen

We got our diagrams there, it’s not great nor awful. Does the job

jenkins_ci1
Alex Atkinson avatar
Alex Atkinson

ty

Alex Atkinson avatar
Alex Atkinson

After trying it out a bit, it really doesn’t seem to be a diagramming tool such much as a mind-mapping UML type solution. Simple actions like searching for an ec2 icon can’t be done. Containers such as vpcs, availability zones, or subnets don’t exist… Anywhere you’re going deeper than the executive level, 30,000 ft viewpoint, probably it’s not a great tool afaict.

sheldonh avatar
sheldonh

Not for cloud diagrams imo. Checkout minigrammer diagrams for cloud. Works well.

sheldonh avatar
sheldonh

Whimsical for flow/decision diagrams is amazing. Great tool to use for decision diagramming and proposals though!

Alex Atkinson avatar
Alex Atkinson

That’s what I’m thinking. That’s their market vertical. Since Mermaid.js there’s really nothing else for tech/science disciplines, though. I’ve even seen the random exec getting excited about it on occasion.

2023-05-17

Alex Atkinson avatar
Alex Atkinson

1000 interns in a room with 1000 laptops will eventually output the complete works of DevOps…. So why is this project so $$$?

bradym avatar

That’s a depressing take.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

I feel like I’m missing some context to this…

this2
managedkaos avatar
managedkaos

perfect fit for #random!

Alex Atkinson avatar
Alex Atkinson

It’s a riff on the Simpsons 1000 monkeys.

Alex Atkinson avatar
Alex Atkinson

I don’t think uplifting was ever the goal there.

2023-05-18

bradym avatar
1
Yonatan Koren avatar
Yonatan Koren

Reminds me of that scene in anchorman

“60% of the time, it works every time”

And I think this applies to observability as well

2023-05-19

sheldonh avatar
sheldonh

Amazing project I’ve been using for a while now. Needs more attention.

It can operate as a terraform,Pulumi, etc manager. Cross platform and blazing fast.

https://aquaproj.github.io/

I just drop into each my Pulumi repos and it’s handled and installs or updates automatically . Highly recommend! Really innovative binary tool manager written Go and dev is amazing. All my ci jobs and local stuff use it so simplified my remote actions as well.

aqua Official Website | aqua

aqua Official Website

2
1
1
1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Looks cool

aqua Official Website | aqua

aqua Official Website

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
aquaproj/aqua

Declarative CLI Version manager written in Go. Support Lazy Install, Registry, and continuous update with Renovate. CLI version is switched seamlessly

sheldonh avatar
sheldonh

It’s solved almost all scripting needs for setting up tools for projects. This + trunk.io and I’m golden!

Matt Gowie avatar
Matt Gowie

Loving trunk.io – one of my team members introduced that and we’re now rolling it out everywhere + the GHA.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Would love to see a demo of trunk, @Matt Gowie

Matt Gowie avatar
Matt Gowie

@Erik Osterman (Cloud Posse) would be happy to – it’s quite simple.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Looks like only a paid solution though, right?

sheldonh avatar
sheldonh

I’ve been using on free tier I don’t use the github app dashboard. I just replaced precommit and other tools with it.

I’ve got my ci lint/test greatly simplified with it. Precommit in ci can be tricky but now all my ci checks are able to run locally.

The actions is great to. I’ve had it do go mod tidy, trigger doc refresh and more. I still codify my work in mage but all the quality checks work great with this tool.

suzuki-shunsuke avatar
suzuki-shunsuke

Hi @sheldonh, thank you for sharing aqua!

2023-05-21

2023-05-22

KrisM avatar

Kubernetes themed Spotify playlist created in parallel over many hours working on Kubernetes clusters. Mostly instrumental and intended for work/coding focus. Enjoy! https://open.spotify.com/playlist/7hB7gMsnAnikpfeuUe3ghQ?si=cc0429b60a9d4241

5

2023-05-23

Hao Wang avatar
Hao Wang

I’ve been in this community for a while, and learned a lot from Cloudposse team. I used Linux for many years, and I hope I can help with some Linux issues. Love tough issues Lol

Hao Wang avatar
Hao Wang

btw just to share that I always suggest my clients to use Cloudposse modules for I can see the high quality codes

2
this1

2023-05-24

Matt Gowie avatar
Matt Gowie

If anybody here wants a free pass to DataDog’s DASH conference in SF in August – Here is a coupon code for a free ticket. Needs to be used before June 6th.

EMHUd2WC

Please reply in thread that you’ve used this coupon code to save other people’s time. I only got the one because my talk didn’t get accepted and I unfortunately can’t attend.

DASH | Scale up, speed upattachment image

DASH is a conference all about building and scaling the next generation of applications, infrastructure, security, and technical teams. Join us for workshops, keynotes, technical sessions, and more.

4

2023-05-25

Lolu Ade avatar
Lolu Ade

Hey all, Looking to see if anyone has any experience with Argo CD Image Updater and Argo CD notifications

venkata.mutyala avatar
venkata.mutyala

Try asking in argocd

digipandit91 avatar
digipandit91

Hey , Do you have junior devops job openings?

2023-05-26

2023-05-28

    keyboard_arrow_up