#random (2019-09)
Non-work banter and water cooler conversation
A place for non-work-related flimflam, faffing, hodge-podge or jibber-jabber you’d prefer to keep out of more focused work-related channels.
Archive: https://archive.sweetops.com/random/
2019-09-02
https://www.youtube.com/watch?v=uDi3zqb3gAA quite old but still worth a watch about how a missing IAM key led to circleCI’s biggest security incident caused by 3rd party provider
2019-09-03
Cloud Conformity highlights violations of AWS and Azure best practices, delivering over 500 different checks across all key areas — security, reliabili…
2019-09-05
anyone use mergify? looks pretty slick… https://mergify.io/
No but have used individual things together to create a similar workflow
so… you still need to have tests, you still need to expose coverage, you still need to provide codestyle
so why not to utilize github protected branching with checks
run all those things in your CI and ask PR to be mergeable only after at least some number of people reviewed the change and all the created tests are passing
^ basically what I had happening. There’s then this github tool you can use that checks if all Statuses have passed and if so merge
Seems like mergify is convenient but you sacrifice ability to perhaps use other tools?
for me it’s a question of do i really want another 3rd party service to access my sourcecode
github.com ok, my ci of choice ok, but github apps doing security scanning, some merge helper tools it’s questionable
we’ve been using dependabot to create prs to test new versions of dependencies. it gets to be a lot of prs. the CI does run all such tests on all such PRs and marks the commit with the status, but then we still have to go in and mark it approved and merge it. just looking for a way to automate that workflow for specific conditions
we mostly just have public projects, so not especially concerned about read access (they already have it). the write access required is a concern though, would be great if github had more granular repo permissions
a scheduled job in you ci which pulls the list of open prs and goes through them to check whether or not it was approved and merge the ones which have tests passed and been approved seems like few liner in python
especially that you can set github in a way that it prevents merging if a pr was not approved by a certain number of people
Yeah sure, just nice to build on something used by a wider community. Decisions around config names and structures get paralyzing, and there are a lot of unknown unknowns for any one person. Having input from lots of users and use cases is huge
in some projects we use whitesource bolt for github to keep an eye on vulns in our dependencies, and the way they do writes to our projects is by creating PRs that are reviewed by us
2019-09-08
Los Angeles: hi all! I hope you will join us this Wednesday at the TestableLA meetup. This time we’re talking about testing Machine Learning, and testing DevOps! https://www.meetup.com/testable/events/wmkmdqyzmbpb/
Wed, Sep 11, 2019, 6:30 PM: TestableLA is all about testing - how to write better tests, and how to make your code easier to test. We welcome talks on a variety of technologies and programming languag
2019-09-09
A Multifunctional Laser Tripwire. Contribute to dekuNukem/daytripper development by creating an account on GitHub.
2019-09-10
Hilarious
Man, if I had an office I’d totally build daytripper~ lol
2019-09-17
Anyone have experience/know of a framework (i’m thinking of something like an API gateway) that allows you to combine responses from 2 different microservices and return to client as a unified response?
An example is having a Post microservice, and a analytics microservice. -The post service returns post information for each post (text, author, title, description … etc) -Analytics service has a different DB and hold analytics related info for each post, e.g. (views, likes, .. etc)
I’m looking for a way, to have a single request from a client to then request information from analytics and post service, join the responses and return to client as aggregate response
I have done some research and found KrakenD but have never heard of it before.
I have Nginx/openresty and some other services running on one VM. Basically VM accepts requests on Openresty and then openresty forwards requests to appropriate service. e.g. below requests getting
Essentially an api / endpoint that hits two different endpoints and maps the results?
yeah
Smells more like a data mapping problem than something a whole framework would do
what are common approaches to solving that problem?
I’ve used a few different libraries for mapping data
but haven’t been super happy with any of them
but it isn’t too hard to roll your own
the simplest approach is to call analytics service from post service, but it would be slower
Well, that’s kind of an interesting question. Is this “middle” layer part of one of the other microservices or is it its own service?
The other question worth asking is why not have the client send two API calls?
^^ true
what language or API framework do the microservices currently use?
what do you mean middle layer?
client -> middle -> Post, Analytics
the client has to hit some api
okay gotcha
It sounds like that API should hit the other two APIs
the frameworks we currently have are java/spring-boot and python/django
so, Django Rest Framework has pretty good mapping support
it can get pretty complicated though
but I’ve set up “remote api” serializers using it
so client -> API 1 -> API 2
we used it to migrate the client over to API 2 with no disruption
but you could use it to do what you are talking about
it’s not out of the box, though
so in your example above, all requests go through API 1 ?
yeah
your example would add a second call
client -> API 1 -> API 2 + API 3
where API 1 is calling both 2 and 3
(and ideally async)
and then join everything together in API 1?
yeah
that’s a way to do it
but it worked well enough
can you see that link?
that’s the code we used for the the base remote serializer
that works with Django Rest Framework
yeah i see that, thank you
but if you aren’t used to DRF
it might not be obvious how to get it all hooked up
but you can see the AuthClient
bit at the top
that is API 1’s interaction class for talking to API 2
ah I see
does managing api changes become difficult?
yes
you are explicitly coupling API 1 to API 2 and 3
and the the client gets coupled to API 1
so going back to asking the question, why not two calls?
i dont know
if the client already knows about API 2 and 3
i didnt think of that
then it probably isn’t worth it to try and combine the calls
unless you are making thousands of them a minute
here’s how you’d end up using the RemoteSerializer: https://gist.github.com/MrHen/6e96f2aa2729446c15ed4ef4189a087b#file-serializers-py
okay so maybe some background. We are somewhat in the phase of migrating to microservices. So authentication is handled by one of those microservices. In the case of 2 requests would we have to have a centralized authentication?
ah, yeah, okay
so you’ll probably want to add the auth calls
directly to the previously existing api
and then once hte auth microservice is up and running
switch the client to hit the auth service
then remove the call between the existing api and the auth service
but if you need to do backend permissions / validation
you might always have a call between the existing api and the auth service
it really depends on the details
but I would say that the end shape should probably be the client making two calls
but that’s just my opinion
we recently made the exact same change in our platform
the RemoteSerializer from above was used to start migrating calls over to the new api
okay so youre saying, (I’m assuming there are 2 APIs: API1 and API2) if API1 handles authentication and API2 does not, then call the auth endpoints from API2 -> API1 when receiving request on API2
?
and if authentication is separated into its own service then API1 would call auth service (API3) and API2 would also call auth service(API3)
yeah, something like that
the details vary case by case
but hopefully that helps a bit
I don’t know that you are going to find a dedicated backend framework that will hook all this up
I do know that Django has some remote auth functionality
Well i found KrakenD which seems to aggregate responses from multiple services
and using a config file for rules
have you ever heard of that?
I haven’t
and also yes, this is very helpful. Thanks
oh, actually, I have seen some stuff like this
usually it’s integrating with external apis
KrakenD seems kind of interesting
I’m not sure I’d want to bake in that dependency though
yeah it seems to do exactly what I want, but i’m worried it could stop being developed
yeah, tough call
like this is the main functionality i really like about it
yeah, looks kind of interesting
I think people tend to be a little too eager to combine api calls, though
I think making multiple calls can be totally fine
it depends on the situation
the best argument for combining calls is related to things like VPCs or protected / hidden endpoints
the best argument against combining calls is that a change to API A will require a change to the client and your KrakenD config
here is the exact situation, which maybe would make sense for a combined call or not. Let me know what you think.
- request to API1 get most recent posts (so you dont know postIds when making initial call from client)
- client has to wait for response (post Ids) from API1 in order to make the call to API2
I do like the 2 requests approach though…. I just dont want it to be too slow
it won’t be slower than combining the two calls
since both calls still have to happen
do you send a single request to API 2 with the post ids?
or are you looping over each post id and calling API 2 for each id?
yeah single request
but when combining you could do them asynchrounously
i guess not a huge gain
you still need the post ids for the second call, right?
yeah you needs post ids for second call
so you still have to wait for the post ids to “resolve”
which is going to be the longest part of the call
ahh yeah youre right
I’d start by trying two calls from the client
and measure how long it takes
since it would be the simplest approach
Awesome, yeah i’m going to start with that.
thanks for your help
no problem!
good luck!
thanks!!
2019-09-21
anyone ever try this out?
Eclipse Vert.x is a tool-kit for building reactive applications on the JVM.
I like what they are preaching over there
the docs are amazing, https://vertx.io/docs/
Eclipse Vert.x is a tool-kit for building reactive applications on the JVM.
2019-09-27
Just in case even 1 person in the 1,411 people here don’t know it: Python 2 end of life is January 1.
Tell your friends. Tell your coworkers at the water cooler. Tell your mailman. Tell your customer you’ve had for 10 years who doesn’t want to change anything. Tell your dog .
Friends don’t let friends use python 2.
The official home of the Python Programming Language
Python 2.7 will retire in 2020. Learn more and see the countdown here.
Except RHEL https://access.redhat.com/solutions/4455511
How is Python 2 supported in RHEL after 2020? How long will Python 2.7 be supported within Red Hat products? As per https://pythonclock.org the Python project is retiring development on Python 2 Jan 1st, 2020. What happens after this date? Python2 is available in an Application stream in RHEL8. Can you let me know what that means for the Python 2 lifecycle within RHEL8? Will Python 2 be available for the entire life of RHEL 8? Will Red Hat continue to provide bugfix/security errata to Python 2 after the EOL of the language?
@Maciek Strömich ^^
But interesting.. I didn’t know that RHEL would support Python 2 until mid 2024.
@Robert considering the speed of doing things in large enterprises/government institutions (some still are forced to use IE6/IE7 ffs ) it’s was kind of obvious that this would happen especially that RHcustomers pay for the support
@Doug Farrell has joined the channel
@Rajesh Babu Gangula has joined the channel
@Sebastian Cavedale has joined the channel
@jacob has joined the channel
2019-09-28
@Nick S has joined the channel
2019-09-29
@daniel has joined the channel
@coreycarvalho has joined the channel
2019-09-30
@Manuel Urbano has joined the channel
@Walter Heck has joined the channel
@Ognen Mitev has joined the channel
@Dipesh Patel has joined the channel