#random (2022-02)
Non-work banter and water cooler conversation
A place for non-work-related flimflam, faffing, hodge-podge or jibber-jabber you’d prefer to keep out of more focused work-related channels.
Archive: https://archive.sweetops.com/random/
2022-02-01
2022-02-02
2022-02-03
Anyone ever author a declarative yaml manifest then need to add some linting/validation to the manifest afterwards? If so, are there any good tools to automate such a thing?
Jsonschema?
It’s supported by all major IDEs, many command line tools, GitHub actions, etc
I was looking at that actually, seems that helm produces a jsonschema from a values file via schema-gen (found the use example here: https://github.com/hashicorp/vault-helm/blob/main/Makefile)
I’ve found this a few weeks ago: https://github.com/23andMe/Yamale
wow, didn’t know about the helm schema-gen subcommand
2022-02-04
2022-02-07
Any conferences people looking forward to this year?
Wondering if I would go to kubecon again. Not sure what to expect with this corona-thingy
2022-02-08
I went to Kubecon in ’21. Even through my mask, the scent of hand sanitizer was heavy in the air.
2022-02-09
2022-02-13
2022-02-14
2022-02-16
Hi Team,
I want to create Azure Container Registry, how should I design it for organization so it will be at best and easy to handle
This Slack is mostly AWS-oriented, but I’m sure someone can help. Have you tried asking in #azure?
hey everyone. Is anyone using serverless-framework? how do you handle the serverless-state.json when you’re deploying in a CI pipeline? shouldn’t cloudformation handle all that when using the aws provider? is there anything similar like handling the state with terraform?
I’m experimenting with AWS SAM currently, but still undecided if it’ll accomplish what I need to it. It’s a mix of a local lambda runner, with options to deploy the lambda via CloudFormation stacks. The core infrastructure of the lambda (VPCs, roles, etc), I still setup using Terraform prior to using SAM, and I just reference those pieces in the SAM configs.
2022-02-17
Hello Team,
I have an instance running on Azure Virtual Machine Scale Set Where it is complaining about installing NuGetBinaries On Ubuntu 18.04 can anyone help with a command on it please. Tried multiple ways to get installed but nothing seemed to be working. Any input please in this issue your response is highly appreciated.
are you following docs from https://docs.microsoft.com/en-us/nuget/install-nuget-client-tools#nugetexe-cli ?
2022-02-18
2022-02-22
Hi all, We are using a webserver with ElasticBeanstalk from 2019., the platform is tomcat 8.5 with java8 running on 64 bit Amazon Linux. Apache as proxy recently (from Jan 30th) we started getting Service Unavailable issues if go to the endpoint from time to time. and if we refresh 2-3 times it will get resolved on its own.
then I download full logs. under elasticbeanstalk-error_log I can see
[Mon Feb 21 10:00:58.338035 2022] [proxy:error] [pid 14882:tid 139757313533696] (13)Permission denied: AH02454: HTTP: attempt to connect to Unix domain socket /var/run/httpd/ (localhost) failed
[Mon Feb 21 10:00:58.338078 2022] [proxy_http:error] [pid 14882:tid 139757313533696] [client <private-ip-here
:12566] AH01114: HTTP: failed to make connection to backend: httpd-UDS, referer: http://<custom-end-point
/1/<name.jsp
?s=sec$$4P!&refresh=300
[Mon Feb 21 10:43:40.663468 2022] [proxy:error] [pid 14882:tid 139757120071424] (13)Permission denied: AH02454: HTTP: attempt to connect to Unix domain socket /var/run/httpd/ (localhost) failed
[Mon Feb 21 10:43:40.663518 2022] [proxy_http:error] [pid 14882:tid 139757120071424] [client <private-ip-here
:21136] AH01114: HTTP: failed to make connection to backend: httpd-UDS
repeated multiple times from Jan30th.
and when I look at access.log
I can see 503 error log exactly at the same time when permission denied error logs in elasticbeanstalk-error_log
And I looked at the running process using ps -aux | grep httpd and ps -aux | grep tomcat
both are running from 2019 and have no restarts.
what more I can do to troubleshoot these issues
thanks
We all know AWS regions have disparity in service hosting. As of today, us-east-1 (N. Virginia) hosts a max of 306 services while ap-northeast-3 (Osaka) has only 127 services hosted. I needed to answer, for any given AWS service which regions host the service, what services are hosted in a given region. Found a few ways. https://www.cloudyali.io/blogs/how-to-find-all-regions-an-aws-service-is-available
2022-02-24
Anyone have any pentesting recommendations for a place that does cryptographic-specific pentests (such as ensuring a company that claims e2e encryption is handling the encryption well)?
if you’re talking about a downstream vendor, this sort of stuff is basically impossible to verify unless they’re willing to hand over all of their infrastructure
This would be for pentesting our own application
2022-02-25
Seems neat, are you using this?
No not yet but I am trying to create POC for my company.
2022-02-26
2022-02-28
@Channel Morning everyone .. Quick Q one of our API is getting loads of hits say e.g. 50k hits in minute, how I can define architecture behind it to handle the load. Please guide.
are you using anything like solace, kafka or rabbitmq etc?
Its a new requirement so I can design best possible solution
RabbitMQ I am aware of
@sohaibahmed98 help me with factors i will have to consider while designing
I think at start you could thing of the following like
- adding cache on some methods which are returning same data.
- check your database queries , try to optimize them and also introduce indexes or compound indexes
- If you not using kafka or rabbitmq then try to move code from synchronous to asynchronous (spring boot) which could be e,g email etc if its not break anything
- use rabbitmq, incase of rabbitmq plz ignore point 3, all asynchronous will be handle by rabbitmq
I am not sure about your current infrastructure its monolith or microservice base
microservices
are u using any cloud?
its on aws fargate or kubernetes?
yes aws kubernetes
which load balancer you are using?
frankly they gave me this statement to demonstrate them what factors I will consider
I thought of load balancer, caching stuff. what else I can consider to serve the purpose?
say as twitter api, it might get 1000s of hits. to serve without fail how should I build my infra so it will help serve the all requests without fail.
- cache
- optimize database
- load balancer
- increase nodes for microservice (auto scalling)
- divide request according to user region,
- use rabbitmq or event driven approach to handle large requests (its important) you can keep track which request failed in order to process again. solace or kafka is super super fast to handle large amount of requests
- use high performance global database or divide then into multi region
also keep in mind request throtling techniques
rate limiting
In one of the project I was getting throttling error so I increased the limit. Is it right thing to do?
did you investigated why you were getting throttling error?
One of the Lambda was keep hitting the API in every few secs
it was internal API though for CI
Also for above question you posted regarding 50k hits. There is lot of guys in this group who are better then me it would be great if they also reply.
Thank you so much for your time, will deep dive into it. It will really helpful to start with. Definitely if anyone knows about it please share your knowledge.
welcome anytime