#release-engineering (2021-06)
All things CI/CD. Specific emphasis on Codefresh and CodeBuild with CodePipeline.
CI/CD Discussions
Archive: https://archive.sweetops.com/release-engineering/
2021-06-01
![sheldonh avatar](https://secure.gravatar.com/avatar/b909e5a82474e9853ff6a6c6111cf0cf.jpg?s=72&d=https%3A%2F%2Fa.slack-edge.com%2Fdf10d%2Fimg%2Favatars%2Fava_0020-72.png)
[thread] :thread: Branching & PR Methodology for Improving Velocity and Reducing Size of PR
I normally use main
with short lived feature branches having used this successfully in a devops oriented role….. see thread for details. Could use some ideas to setup remote team for better success…..
![sheldonh avatar](https://secure.gravatar.com/avatar/b909e5a82474e9853ff6a6c6111cf0cf.jpg?s=72&d=https%3A%2F%2Fa.slack-edge.com%2Fdf10d%2Fimg%2Favatars%2Fava_0020-72.png)
In my new role working with a backend development team, I’m helping design a process to improve velocity of getting changes in, but also reducing the size of PR’s. Policy dictates that myself and another dev have to review anything from an offshore team for security.
Current PR size can be thousands of lines and files, so I want to change this to more trunk based development (especially per pre-release of product). I need to configure the merge policy as well.
I have 2 initial thoughts:
- My preferred/method I know the best is simple trunk based development with short lived feature branches. To reduce complexity in reviews, require testing and linting hook checks to pass prior to review. Pros: Faster integration to move towards production, less size in PR’s is a better review experience and more manageable. Cons: Far more PR’s to review from other team.
- Due to the noise this would generate in PR count, I’m not sure this would be viable, so second would be to use
main
<—develop
but I’m used to rebase/squash workflow to keep history and PR’s very targeted and simple and unclear if I’m going to be dealing with a lot more noise in the end trying to merge sets of changes fromdevelop
tomain
at that point. Not sure if can use rebase workflow with a long-lived branch like this so main will end up being different than develop with merge commits and less atomic history I feel. Pros: less PR’s generated, most devs know a workflow similar to this Cons: extra complexity with 2 main branches, probably larger PR’s more likely, slower to integrate into main.
Feels like I’m kinda making a decision for more PR noise, in smaller PR’s but a ton more to review, or less PR’s and more complexity to review… agree? I tend to think more PR’s but smaller is less context switching and cost overall.
![loren avatar](https://secure.gravatar.com/avatar/d1e25dcfbc68a0857a04dd78c9afe952.jpg?s=72&d=https%3A%2F%2Fa.slack-edge.com%2Fdf10d%2Fimg%2Favatars%2Fava_0003-72.png)
We started with #2 for some projects, and #1 for others, and eventually switched to #1 for everything. We just didn’t see any benefits from #2, and it complicated releases/deployments unnecessarily. #1 with a solid CI/CD pipeline is so easy and refreshing, definitely an accelerator and confidence booster
![sheldonh avatar](https://secure.gravatar.com/avatar/b909e5a82474e9853ff6a6c6111cf0cf.jpg?s=72&d=https%3A%2F%2Fa.slack-edge.com%2Fdf10d%2Fimg%2Favatars%2Fava_0020-72.png)
Agree @loren. I think #1 is perfect as well. I guess I need to get better clarity on what “compliance” means truly to stop guessing of what really has to be reviewed at what point as maybe that’s the core driver.
Adding more layers into the review process if it’s not literally shipped concerns me. Instead, couldn’t it deploy to staging from main and finally a pending approval of all these changes in Azure Pipelines as a deployment task? At that point the included changes maybe could be reviewed, but not each individual merge into main before that. Maybe I can shift the “approval” concept to the release itself.
![bradym avatar](https://avatars.slack-edge.com/2023-06-21/5464816405572_dd21bed1bf537acb6539_72.jpg)
We used to use #2 for everything, now we use #1 for everything and it’s much better. Less confusing and easier to manage.
It won’t magically shrink your PRs, though. I still occasionally hear our devs talking about a monster PR, so you’ll need to couple this with something else to motivate/dictate smaller PRs.
![Alex Jurkiewicz avatar](https://avatars.slack-edge.com/2020-09-08/1346106958085_9b44ddacd6267cc803c8_72.jpg)
Yep, another vote for #1. I think the key is that you need a way to encourage small PRs and discourage large PRs.
That means get smaller PRs merged easily without stress.
- fast PR process – fast CI loop & review response time
- good protection against merging broken code – CI is complete. No need for manual QA / regression tests.
- developers have confidence their changes will not disrupt others. Features flags or a similar system with a culture of using them. Developers can launch totally empty skeletons of functionality to prod and feel confident they can enable it purely for themselves, or only in test environment even if the code ships to prod
![Alex Jurkiewicz avatar](https://avatars.slack-edge.com/2020-09-08/1346106958085_9b44ddacd6267cc803c8_72.jpg)
Larger PRs get pushback, and review comments like “can you break out these changes to a new small PR?”
![Michael Warkentin avatar](https://secure.gravatar.com/avatar/219e6891cd29919cfad373a72eca17dd.jpg?s=72&d=https%3A%2F%2Fa.slack-edge.com%2Fdf10d%2Fimg%2Favatars%2Fava_0024-72.png)
+1 to feature flags to allow for easier incremental release of functionality
![sheldonh avatar](https://secure.gravatar.com/avatar/b909e5a82474e9853ff6a6c6111cf0cf.jpg?s=72&d=https%3A%2F%2Fa.slack-edge.com%2Fdf10d%2Fimg%2Favatars%2Fava_0020-72.png)
Any articles on simplifying this? I don’t need to be sold, I’ve done this before. However, with a remote team requiring review of all changes I’m going to need to properly plan this out and sell it to show the overall value actually improves the PR review process and complexity even though it requires more reviews.
![Michael Warkentin avatar](https://secure.gravatar.com/avatar/219e6891cd29919cfad373a72eca17dd.jpg?s=72&d=https%3A%2F%2Fa.slack-edge.com%2Fdf10d%2Fimg%2Favatars%2Fava_0024-72.png)
A couple of presentations:
• https://speakerdeck.com/charity/cd
• https://speakerdeck.com/sullis/small-batch-deployments-2019-08-10
![attachment image](https://files.speakerdeck.com/presentations/d03812c0daa64e69a1ceae8bd06068a8/slide_0.jpg?17765227)
Why your software should be auto-deployed within 15 minutes after you merge it, with no manual gates. This is the key to high performing teams and high-quality software.
![sheldonh avatar](https://secure.gravatar.com/avatar/b909e5a82474e9853ff6a6c6111cf0cf.jpg?s=72&d=https%3A%2F%2Fa.slack-edge.com%2Fdf10d%2Fimg%2Favatars%2Fava_0020-72.png)
Any articles on how this might apply when needing to “audit” the incoming changes so the production release is marked as fully code reviewed? I was thinking that decoupling that “audit” as the final signoff coming during the release pipeline and review would be a way to control and confirm all changes since last release. This would eliminate PR being the bottleneck for every review and only production release being the thing to work on next
![Harry avatar](https://secure.gravatar.com/avatar/0cd5f2112ac91e49296b221b7adb58f3.jpg?s=72&d=https%3A%2F%2Fa.slack-edge.com%2Fdf10d%2Fimg%2Favatars%2Fava_0014-72.png)
We’re using #2 at the moment and I’m feeling the pain. I definitely want to get to #1 but we don’t have enough test coverage to deploy changes without a manual regression test. As soon as I can get some more pieces in place I’m going to be moving us to #1. I’ll also echo what someone already mentioned that feature switches are key for getting smaller PRs. I’ve not quite got to the point where anything over 1000 lines is automatically closed (ignoring any autogenerated files) but I’m tempted every time I see a large PR taking over a week to ship.
![sheldonh avatar](https://secure.gravatar.com/avatar/b909e5a82474e9853ff6a6c6111cf0cf.jpg?s=72&d=https%3A%2F%2Fa.slack-edge.com%2Fdf10d%2Fimg%2Favatars%2Fava_0020-72.png)
Any articles or tips on implementing feature switches to a team that hasn’t done this and needs an easy button? Most of the products are mostly still in pre-release phase too so might be less complex right now
![Michael Warkentin avatar](https://secure.gravatar.com/avatar/219e6891cd29919cfad373a72eca17dd.jpg?s=72&d=https%3A%2F%2Fa.slack-edge.com%2Fdf10d%2Fimg%2Favatars%2Fava_0024-72.png)
I’m sure https://launchdarkly.com would have some blog content on that
![attachment image](https://launchdarkly.com/img/social_logo.jpg)
LaunchDarkly provides simple, scalable feature flag & toggle management (feature management) for the modern enterprise. Eliminate risk, deliver value.
2021-06-02
2021-06-14
![sunnybook8 avatar](https://secure.gravatar.com/avatar/128181a4ff0818275f4092e1fa516a2f.jpg?s=72&d=https%3A%2F%2Fa.slack-edge.com%2Fdf10d%2Fimg%2Favatars%2Fava_0016-72.png)
Just want to get some feedback from everyone.
- we have a python backend and a react front. currently everything is in aws. my architect recommended this setup. What do you think? Is this an easy task to do in aws?
- Does anyone have experience with Trend Micro Cloud Conformity. Do you recommend it or not?
- For SCA and SAST tools, what would be good libraries or tools for python and react code base?
![kareem.shahin avatar](https://secure.gravatar.com/avatar/e3523be2d3654c14fff5c08f953e9fc7.jpg?s=72&d=https%3A%2F%2Fa.slack-edge.com%2Fdf10d%2Fimg%2Favatars%2Fava_0015-72.png)
Can’t recommend a particular SAST tool for either but take a look at OWASP SAST list https://owasp.org/www-community/Source_Code_Analysis_Tools
Source Code Analysis Tools on the main website for The OWASP Foundation. OWASP is a nonprofit foundation that works to improve the security of software.
![sunnybook8 avatar](https://secure.gravatar.com/avatar/128181a4ff0818275f4092e1fa516a2f.jpg?s=72&d=https%3A%2F%2Fa.slack-edge.com%2Fdf10d%2Fimg%2Favatars%2Fava_0016-72.png)
yeah I also stumbled this on google the other day. very helpful.
2021-06-16
2021-06-18
2021-06-23
![mfridh avatar](https://secure.gravatar.com/avatar/bc28bf133c2c735cf9e62952c4965389.jpg?s=72&d=https%3A%2F%2Fa.slack-edge.com%2Fdf10d%2Fimg%2Favatars%2Fava_0000-72.png)
I know for sure I’ve seen a good tool for this before:
Tracking checksums (or content) of remote URLs/files? (Most cases it’s probably git repos). For example to refuse build until you’ve accepted the new checksum of upstream or so… thoughts?
![Blaise Pabon avatar](https://secure.gravatar.com/avatar/9db34d0c21fdaf687b05eff5422bd7cb.jpg?s=72&d=https%3A%2F%2Fa.slack-edge.com%2Fdf10d%2Fimg%2Favatars%2Fava_0014-72.png)
Oh, god, does this bring back memories…. I had to build something like this a while back because we were not mapping the source code commit IDs to the MD5 ids of the build ISO files…. @mfridh, ideally this would be built into your artifact repository (eg. Artifactory, Nexus, etc.) If you don’t have one, it’s a lot easier to get one and then decide whether to DIY later….