#random (2023-05)
Non-work banter and water cooler conversation
A place for non-work-related flimflam, faffing, hodge-podge or jibber-jabber you’d prefer to keep out of more focused work-related channels.
Archive: https://archive.sweetops.com/random/
2023-05-01
Hi Folks, I am Sudhish - https://www.sudhishkr.com/ - and I am a Software Engineer by profession from San Francisco. I have worked extensively in the Platform/Infra/Data side in the last decade. Looking fwd to the learning.
On the side, I do a lot of consulting. Checkout mytytech.com - and we would be happy to help you out.
Sudhish KR’s personal website
2023-05-04
Wow. This Krakend (api gw) guy does an amazing job of showing off their kit. https://www.youtube.com/watch?v=VtXXZRO84t8
I feel like Kong used to have content a lot like this, but it’s hard to find now. Did they prune it to posture their enterprise offering more prominently? IDK. Anyway, Krakend looks sick. But what are folks around here using lately?
2023-05-07
Please reach out and/or check out our website, www.southlakeconsulting.com!
2023-05-08
I’m trying to find some of the first/early references to a multi-environment product pipeline. Does anyone have a line on something like this? I found this interesting doc sponsored by the US Army Computer Systems Command and The IEEE Computer Society that shows:
• The Software Life Cycle Management Modle (Fig. 1)
• We’re still struggling with the same qa/ops concerns as ever… Mostly. (Figure 1. Factors in Software Quality) But it doesn’t explicitly state the definition of environment, or align it with “dev, qa, stag, prod…”.
Here’s a paper from 1990 that starts to talk about the SDLC, and testing. Interesting read, if short. https://eli.johogo.com/pdf/jsm-90.pdf
… I’ll bread crum into the first paper I find that specifically defines what I’m looking for.
The Handbook of Software Quality Assurance Techniques Applicable to the Nuclear Industry (1987) has the first graph showing cost to remediate an issue vs delivery phase that I know of on page 10.2. This paper reads a lot like most compliance frameworks… https://inis.iaea.org/collection/NCLCollectionStore/_Public/19/018/19018784.pdf
HITACHI: PIONEERING A “FACTORY” STRATEGY AND STRUCTURE FOR LARGE-SCALE SOFTWARE DEVELOPMENT Page 18: “Around 1970 we came to believe that we had to introduce a components control system similar to what you find for hardware…” Page 21: Staffing allocation: 55% Planning & Design, 5% coding, 35% debugging, 10% inspection….. Page 32: The broke Brooks Law… Adding people to deliver on time worked? WHAT? https://dspace.mit.edu/bitstream/handle/1721.1/48057/hitachipioneerinx00cusu.pdf
Then there’s a bunch of talk about mitigating “merge hell” with Git vs SVN… I remember that. And Agile comes along with DevOps. I went from Ops (Microsoft bs) to DevOps (linux yay) when devops became a thing… It seems I took the devops pipeline for granted, having started with this classic.
hey y’all! Trying to sign up for your newsletter, but the form seems broken. Any ideas what’s up?
@Erik Osterman (Cloud Posse)
Hey @Leah Blank thanks for letting me know. I’ll check
2023-05-10
Does anyone use opensource kong with open source openid connect baked in, instead of paying for Enterprise to get the feature? I found this repo that looks promising, but wonder what other folks are using for api gw these days. https://github.com/revomatico/docker-kong-oidc There’s Tyk, which is open source with all features, including oidc… https://github.com/TykTechnologies/tyk
Apisix is a good option for open source api gateway. https://github.com/apache/apisix/blob/master/docs/en/latest/plugins/openid-connect.md
title: openid-connect
keywords:
• Apache APISIX
• API Gateway
• OpenID Connect
• OIDC
description: OpenID Connect allows the client to obtain user information from the identity providers, such as Keycloak, Ory Hydra, Okta, Auth0, etc. API Gateway APISIX supports to integrate with the above identity providers to protect your APIs.
Description
OpenID Connect (OIDC) is an authentication protocol based on the OAuth 2.0. It allows the client to obtain user information from the identity provider (IdP), e.g., Keycloak, Ory Hydra, Okta, Auth0, etc. API Gateway Apache APISIX supports to integrate with the above identity providers to protect your APIs.
Attributes
NOTE: encrypt_fields = {"client_secret"}
is also defined in the schema, which means that the field will be stored encrypted in etcd. See encrypted storage fields.
Scenarios
:::tip
Tutorial: Use Keycloak with API Gateway to secure APIs
:::
This plugin offers two scenorios:
- Authentication between Services: Set
bearer_only
totrue
and configure theintrospection_endpoint
orpublic_key
attribute. In this scenario, APISIX will reject requests without a token or invalid token in the request header. - Authentication between Browser and Identity Providers: Set
bearer_only
tofalse.
After successful authentication, this plugin can obtain and manage the token in the cookie, and subsequent requests will use the token. In this mode, the user session will be stored in the browser as a cookie and this data is encrypted, so you have to set a key for encryption viasession.secret
.
Token introspection
Token introspection validates a request by verifying the token with an OAuth 2.0 authorization server.
You should first create a trusted client in the identity server and generate a valid JWT token for introspection.
The image below shows an example token introspection flow via a Gateway:
The example below shows how you can enable the Plugin on Route. The Route below will protect the Upstream by introspecting the token provided in the request header:
curl <http://127.0.0.1:9180/apisix/admin/routes/5> -H 'X-API-KEY: edd1c9f034335f136f87ad84b625c8f1' -X PUT -d '
{
"uri": "/get",
"plugins":{
"openid-connect":{
"client_id": "${CLIENT_ID}",
"client_secret": "${CLIENT_SECRET}",
"discovery": "${DISCOVERY_ENDPOINT}",
"introspection_endpoint": "${INTROSPECTION_ENDPOINT}",
"bearer_only": true,
"realm": "master",
"introspection_endpoint_auth_method": "client_secret_basic"
}
},
"upstream":{
"type": "roundrobin",
"nodes":{
"httpbin.org:443":1
}
}
}'
Now, to access the Route:
curl -i -X GET <http://127.0.0.1:9080/get> -H "Host: httpbin.org" -H "Authorization: Bearer {JWT_TOKEN}"
In this example, the Plugin enforces that the access token and the Userinfo object be set in the request headers.
When the OAuth 2.0 authorization server returns an expire time with the token, it is cached in APISIX until expiry. For more details, read:
- lua-resty-openidc’s documentation and source code.
exp
field in the RFC’s Introspection Response section.
Introspecting with public key
You can also provide the public key of the JWT token for verification. If you have provided a public key and a token introspection endpoint, the public key workflow will be executed instead of verification through an identity server. This is useful if you want to reduce network calls and speedup the process.
The example below shows how you can add public key introspection to a Route:
curl <http://127.0.0.1:9180/apisix/admin/routes/5> -H 'X-API-KEY: edd1c9f034335f136f87ad84b625c8f1' -X PUT -d '
{
"uri": "/get",
"plugins":{
"openid-connect":{
"client_id": "${CLIENT_ID}",
"client_secret": "${CLIENT_SECRET}",
"discovery": "${DISCOVERY_ENDPOINT}",
"bearer_only": true,
"realm": "master",
"token_signing_alg_values_expected": "RS256",
"public_key": "-----BEGIN PUBLIC KEY-----
{public_key}
-----END PUBLIC KEY-----"
}
},
"upstream":{
"type": "roundrobin",
"nodes":{
"httpbin.org:443":1
}
}
}'
In this example, the Plugin can only enforce that the access token should be set in the request headers.
Authentication through OIDC Relying Party flow
When an incoming request does not contain an access token in its header nor in an appropriate session cookie, the Plugin can act as an OIDC Relying Party and redirect to the authorization endpoint of the identity provider to go through the OIDC authorization code flow.
Once the user has authenticated with the identity provider, the Plugin will obtain and manage the access token and further interaction with the identity provider. The access token will be stored in a session cookie.
The example below adds the Plugin with this mode of operation to the Route:
curl <http://127.0.0.1:9180/apisix/admin/routes/5> -H 'X-API-KEY: edd1c9f034335f136f87ad84b625c8f1' -X PUT -d '
{
"uri": "/get",
"plugins": {
"openid-connect": {
"client_id": "${CLIENT_ID}",
"client_secret": "${CLIENT_SECRET}",
"discovery": "${DISCOVERY_ENDPOINT}",
"bearer_only": false,
"realm": "master"
}
},
"upstream": {
"type": "roundrobin",
"nodes": {
"httpbin.org:443": 1
}
}
}'
In this example, the Plugin can enforce that the access token, the ID token, and the UserInfo object to be set in the request headers.
Troubleshooting
- If APISIX cannot resolve/connect to the identity provider (e.g., Okta, Keycloak, Authing), check/modify the DNS settings in your configuration file (
conf/config.yaml
). - If you encounter the error
the error request to the redirect_uri path, but there's no session state found,
please confirm whether the currently accessed URL carriescode
andstate,
and do not directly accessredirect_uri.
- If you encounter the error
the error request to the redirect_uri path, but there's no session state found
, please check theredirect_uri
attribute : APISIX will initiate an authentication request to the identity provider, after the authentication service completes the authentication and authorization logic, it will redirect to the address configured byredirect_uri
(e.g.,<http://127.0.0.1:9080/callback
>) with ID Token and AccessToken, and then enter APISIX again and complete the function of token exchange in OIDC logic. Theredirect_uri
attribute needs to meet the following conditions:
• redirect_uri
needs to be captured by the route where the current APISIX is located. For example, the uri
of the current route is /api/v1/*
, redirect_uri
can be filled in as /api/v1/callback
;
• scheme
and host
of redirect_uri
(scheme:host
) are the values required to access APISIX from the perspective of the identity provider.
Or you can use traefik as well.
If anyone wants to join a quick webinar I’m helping panel on secrets starting at 19:00 UTC (20m from now)
Not a product pitch — will be talking about k8s secrets operators, CI secrets, etc
2023-05-12
Hi all .. I have a question regarding a small yet important PR that I opened on Github for one of the repos and it’s kinda urgent .. can someone help with that and the review eventually?
2023-05-14
How many full-time Platform Engineers / Cloud Engineers / DevOps Engineers are required for a cloud migration, where there are several production environments (k8s clusters, database and blob storage services, etc)
Say AWS to GCP
Just wondering what people would place this number at (random ballpark #)
I guess that would mean full time on that project and nothing else, right?
oh yeah, may take 1 month to 3
The number of engineers needed and how long it takes will depend entirely on your environment, how you are using AWS, how you plan to use GCP, how familiar your engineers are with both AWS and GCP (and the fact that this will likely vary significantly from one engineer to another), how directly services can be translated, how you’re running your applications, what gotchas are encountered along the way and many more factors.
Figuring it out should be a project itself.
Figuring it out should be a project itself.
Well said
2023-05-15
2023-05-16
Is whimsical good for cloud architecture diagrams? I can’t see anything on the site that says it would be, but…. ?
If you can use it for wireframes you can use it for pretty much anything…
But hopefully there’s an option to keep all of your AWS service logos saved instead of having to copy and paste each time you make a new diagram
ty
After trying it out a bit, it really doesn’t seem to be a diagramming tool such much as a mind-mapping UML type solution. Simple actions like searching for an ec2 icon can’t be done. Containers such as vpcs, availability zones, or subnets don’t exist… Anywhere you’re going deeper than the executive level, 30,000 ft viewpoint, probably it’s not a great tool afaict.
Not for cloud diagrams imo. Checkout minigrammer diagrams for cloud. Works well.
Whimsical for flow/decision diagrams is amazing. Great tool to use for decision diagramming and proposals though!
That’s what I’m thinking. That’s their market vertical. Since Mermaid.js there’s really nothing else for tech/science disciplines, though. I’ve even seen the random exec getting excited about it on occasion.
2023-05-17
1000 interns in a room with 1000 laptops will eventually output the complete works of DevOps…. So why is this project so $$$?
That’s a depressing take.
perfect fit for #random!
It’s a riff on the Simpsons 1000 monkeys.
I don’t think uplifting was ever the goal there.
2023-05-18
Reminds me of that scene in anchorman
“60% of the time, it works every time”
And I think this applies to observability as well
2023-05-19
Amazing project I’ve been using for a while now. Needs more attention.
It can operate as a terraform,Pulumi, etc manager. Cross platform and blazing fast.
I just drop into each my Pulumi repos and it’s handled and installs or updates automatically . Highly recommend! Really innovative binary tool manager written Go and dev is amazing. All my ci jobs and local stuff use it so simplified my remote actions as well.
aqua Official Website
Declarative CLI Version manager written in Go. Support Lazy Install, Registry, and continuous update with Renovate. CLI version is switched seamlessly
It’s solved almost all scripting needs for setting up tools for projects. This + trunk.io and I’m golden!
Loving trunk.io – one of my team members introduced that and we’re now rolling it out everywhere + the GHA.
Would love to see a demo of trunk, @Matt Gowie
@Erik Osterman (Cloud Posse) would be happy to – it’s quite simple.
Looks like only a paid solution though, right?
I’ve been using on free tier I don’t use the github app dashboard. I just replaced precommit and other tools with it.
I’ve got my ci lint/test greatly simplified with it. Precommit in ci can be tricky but now all my ci checks are able to run locally.
The actions is great to. I’ve had it do go mod tidy, trigger doc refresh and more. I still codify my work in mage but all the quality checks work great with this tool.
Hi @sheldonh, thank you for sharing aqua!
2023-05-21
2023-05-22
Kubernetes themed Spotify playlist created in parallel over many hours working on Kubernetes clusters. Mostly instrumental and intended for work/coding focus. Enjoy! https://open.spotify.com/playlist/7hB7gMsnAnikpfeuUe3ghQ?si=cc0429b60a9d4241
2023-05-23
I’ve been in this community for a while, and learned a lot from Cloudposse team. I used Linux for many years, and I hope I can help with some Linux issues. Love tough issues Lol
btw just to share that I always suggest my clients to use Cloudposse modules for I can see the high quality codes
2023-05-24
If anybody here wants a free pass to DataDog’s DASH conference in SF in August – Here is a coupon code for a free ticket. Needs to be used before June 6th.
EMHUd2WC
Please reply in thread that you’ve used this coupon code to save other people’s time. I only got the one because my talk didn’t get accepted and I unfortunately can’t attend.
DASH is a conference all about building and scaling the next generation of applications, infrastructure, security, and technical teams. Join us for workshops, keynotes, technical sessions, and more.
2023-05-25
Hey all, Looking to see if anyone has any experience with Argo CD Image Updater and Argo CD notifications
Try asking in argocd
Hey , Do you have junior devops job openings?