#general (2024-04)
General conversations related to DevOps/Automation
General Discussions
2024-04-01
Hey everyone, give a warm welcome to our newest members!
- @Yili Kerim
- @alla2
Good to have you here =)
22024-04-02
Hey everyone, give a warm welcome to our newest members!
- @Jay
- @paparao.prema75
- @Ritika Kumar
Good to have you here =)
42024-04-03
Hey everyone, give a warm welcome to our newest members!
- @Jessie Puls
- @Gobi Dasu
- @umuhammad97
- @Wendell
- @jbontech
- @djaoas
Good to have you here =)
22024-04-04
Hey everyone, give a warm welcome to our newest members!
- @Andrey Klyukin
- @Dale
- @mikaelec
Good to have you here =)
22024-04-05
#general Not sure I can post my query here. Still looking for a solution.
I am trying to migrate a database from AWS RDS to GCP, the size is around 120GB. I thought it would be fast if I create a dump file and copy it to GCP and restore it. But while doing that it’s taking almost 7hours time. Is there any other approach that I can anticipate this very fast? Any suggestions?.
Have you eliminated CPU/IO bottlenecks as the cause?
How are you transferring the dump?
Are you transferring and loading it at the same time? e.g. piping into the database as your transferring, or simply transferring the dump itself is taking 7 hours.
To speed up the transfer of the dump, see about parallelizing the transfer. One way to do that without much extra tooling is to just use the split command to chunk it on lines, then use rsync to download files in parallel. On the other end you can reassemble the files.
Hello @Erik Osterman (Cloud Posse) thanks for the update.. Sorry i was travelling, so couldnt look at your reply. Transferring file took an hour only, dump restore is taking 7hours time..
I could upgrade machine type to achieve high network bandwith which reduced 3hours. But still its pretty high for the critical application migration. Do you still recommend to use the split and rsync will help here??
Ok, so looks like “pay to play” will have the highest ROI. You can resize the database instance after migration.
Just by upgrading the machine type you achieved greater than a 50% reduction in time.
The other thing you can do is disable indexes on load, and add those afterwards.
Since the transfer time is an hour, it depends on how much time you have to prepare for the lift & shift. You’ll continue to get modest gains by splitting the dump and transferring it in parallel and reassembling on the other side.
I would first try increasing the size of the instance even more, just to prove a point. There’s a tradeoff between the cost of people time and machine time, and opportunity cost. The longer it takes to find the solution, the longer the overall time to complete the migration will take (not just the time to ETL the data). So if you receive push back, find out what the true concerns are.
The overall rule-of-thumb advice we give our customers for migrations is the standard (good, fast, cheap). Pick 2.
Hey everyone, give a warm welcome to our newest members!
- @Adel Hashem
- @Vamshi Mylaram
- @tfji
- @Ben Rosand
- @Ben
- @Samuel Frimpong
- @Justin
Good to have you here =)
2024-04-06
Hey everyone, give a warm welcome to our newest members!
- @Justine Devasia
- @career.bash016
Good to have you here =)
22024-04-07
Hey everyone, give a warm welcome to our newest members!
- @Deivids Egle
Good to have you here =)
22024-04-08
Hey everyone, give a warm welcome to our newest members!
- @Halo Hsu
Good to have you here =)
22024-04-09
Hey everyone, give a warm welcome to our newest members!
- @Msk
- @NXT1
Good to have you here =)
12024-04-10
Hey everyone, give a warm welcome to our newest members!
- @Bolt17
Good to have you here =)
2
Hello #general, I am setting a CloudRun service and want to pass some environment variables. I could configure it in the CloudRun console under variables and secrets and pass a static value. But my requirement is i want read the variable from the secretsmanager. So the secretname should be passed in the variable section but the value should be retried from secretsmanager. I couldn’t find any references to read the variable from secretsmanager. Can someone help me if have suggestions..
I suggest using #gcp so you reach that audience.
2024-04-11
Hey everyone, give a warm welcome to our newest members!
- @linear_asks
- @amnesia-allow0f
- @miko
- @Atmosly Platform engineering
- @Tony
- @Jason G
- @Simon Heather
Good to have you here =)
4
2024-04-12
Hey everyone, give a warm welcome to our newest members!
- @MYacineK
- @Aditya PR
- @Alex Berber
- @Aditya Pandhikanda Ramesh
Good to have you here =)
22024-04-13
Hey everyone, give a warm welcome to our newest members!
- @Julio Perez
- @dropinboy
- @Ben
- @tamish60
- @ChrisP
- @Carl Anders Düvel
Good to have you here =)
52024-04-14
gotta test run 100+ variations of a deployment (terraform infra + ansible provisioning). what would you guys use to automate this?
Hey everyone, give a warm welcome to our newest members!
- @nick
- @nrs-status
Good to have you here =)
32024-04-15
Hey everyone, give a warm welcome to our newest members!
- @microgag
- @Maks
- @Stephan Helas
- @Shelley
Good to have you here =)
32024-04-16
Hey everyone, give a warm welcome to our newest members!
- @Tom Bee
- @Amos Gewirtz
- @Andrew Chemis
- @Eddie N.
Good to have you here =)
22024-04-17
Hey everyone, give a warm welcome to our newest members!
- @Krishna
- @Aniket
- @Saksham
- @Mark Brooks
- @Jmarz
- @Michael Klatskin
- @Zack
Good to have you here =)
222024-04-18
Hey everyone, give a warm welcome to our newest members!
- @Dinko Mitic
- @Matt
- @davidjaffie
- @William
- @Maxime
Good to have you here =)
32024-04-19
Hey everyone, give a warm welcome to our newest members!
- @emitor
Good to have you here =)
112024-04-20
Hey everyone, give a warm welcome to our newest members!
- @Lloyd Chang
Good to have you here =)
112024-04-21
Hey everyone, give a warm welcome to our newest members!
- @David C
Good to have you here =)
12024-04-22
Hey everyone, give a warm welcome to our newest members!
- @Bernie
- @Marvin de Bruin
- @Steffan Norberhuis
- @Ashwin Reddy
- @Stanley Tso
- @tony384
Good to have you here =)
3
Unsure where to post this. In the newsletter, i cannot see it clearly on mobile because the formatting of the image shows up on the left of the text giving the text room enough for about one word per line
It makes it challenging to read
I’m using the stock android gmail client
I agree.. Need to change platform i think.
I made some tweaks, let me know if Monday’s newsletter is better.
Looks much better
2024-04-23
Hey everyone, give a warm welcome to our newest members!
- @Prasanth
- @Leonie JM
Good to have you here =)
22024-04-24
Hello, team!
Hey everyone, give a warm welcome to our newest members!
- @rahulbhatu1991
- @Maurysnake
- @Coby
Good to have you here =)
22024-04-25
Been a little while, how is everyone holding up?
Wow, it sure has been!
Haven’t seen you around on #office-hours
Hey @Erik Osterman (Cloud Posse)! I’ve been a little preoccupied at AWS, working on an upcoming AWS Native Service!
I still wish I had time to contribute, and join the office hours!
Hey everyone, give a warm welcome to our newest members!
- @King
- @natali.fpaura
- @Patrick Berrigan
- @Jason
- @Joe Lauzze
Good to have you here =)
12
2024-04-26
Hey everyone, give a warm welcome to our newest members!
- @James
- @Vikash
Good to have you here =)
312024-04-27
Hey everyone, give a warm welcome to our newest members!
- @Dhruv Tiwari
- @James Berger - 3Play Media
Good to have you here =)
32024-04-28
Hey everyone, give a warm welcome to our newest members!
- @Emerson H
- @Max
Good to have you here =)
12024-04-29
Hey everyone, give a warm welcome to our newest members!
- @Noam Guy
- @Ronie
Good to have you here =)
32024-04-30
Hey everyone, give a warm welcome to our newest members!
- @andrea
- @rjshk013
Good to have you here =)
1