#ops (2019-11)

Archive: https://archive.sweetops.com/ops/

2019-11-20

ols avatar

Interested in thoughts on the TSB outage report if anyone has read it yet: https://www.tsb.co.uk/news-releases/slaughter-and-may/

2019-11-25

Janis Peisenieks avatar
Janis Peisenieks

Hi everyone! This seems like the most appropriate channel to ask for help. Does anyone have any experience setting up user-content domains? I.e. githubusercontent.com ?

Janis Peisenieks avatar
Janis Peisenieks

We have set ours up, but it looks like it’s being somehow magically throttled/blocked by some ISP’s/Enterprise firewalls and the like. And I’m curious whether someone knows how to go about making sure that a domain like that doesn’t get blocked as suspicious

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Do you mean like hotlinking to [raw.githubusercontent.com](http://raw.githubusercontent.com)?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

(for this you need a proxy)

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

…it’s not intended to be used as a CDN

Janis Peisenieks avatar
Janis Peisenieks

No, no, what I mean is that we have our own CDN (i.e. janis-user-content.com) that serves the content our users upload. Basically a user-upload sandbox domain to make sure it doesn’t share any cookies with our main domain. We then serve all user-content from that CDN/Domain inside of our app. Well some Enterprise firewalls seem to not trust our user-content domain and sporadically block it. I’m wondering if someone else has any tips/tricks on how to make the domain look… less suspicious?

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

@Janis Peisenieks the domain name itself should not be suspicious to any enterprise firewalls unless it’s explicitly blocked (blacklisted)

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

it must be something else, e.g. headers, security settings, etc.

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

for example, take a look at Content-Security-Policy header, it can block many resources from loading on a site

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
  'Content-Security-Policy': 'frame-src <https://aaaaaaa.com>; default-src \'none\'; img-src \'self\' data: blob: *; script-src \'self\' blob: <https://aaaaaaaa.org>; style-src \'unsafe-inline\'; object-src \'none\'; frame-ancestors \'self\'; connect-src \'self\' <https://aaaaaa.com/>; base-uri \'self\'; form-action \'self\'; manifest-src \'self\'');
Janis Peisenieks avatar
Janis Peisenieks

The main problem here is that it’s nowhere near consistent, and seems to happen sporadically for some users, but usually quite a few

Janis Peisenieks avatar
Janis Peisenieks

which makes me think that there is something wrong (or at least not right) with our domain

Janis Peisenieks avatar
Janis Peisenieks

Which makes me think that it shouldn’t be Content-Security-Policy realted

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

so, since Content-Security-Policy could be diff on each site, the behavior could be different

Janis Peisenieks avatar
Janis Peisenieks

hmm

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

also, do you use SSL to load assets from your domain?

Janis Peisenieks avatar
Janis Peisenieks

Yes

Janis Peisenieks avatar
Janis Peisenieks

And the weird thing is that we always serve the user-content from CDN from our main domain. That part doesn’t change, which (if I understand CSP correctly) means that CSP shouldn’t be changing as well. At least we don’t change it in any way

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

take a look at headers and compare them on working and not working sites

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

something should be diff there

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
<https://medium.com/@tom.cook/edge-lambda-cloudfront-custom-headers-3d134a2c18a2>
<https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Security-Policy/default-src>
<https://content-security-policy.com>
Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

these are just some of the headers that could affect it

Janis Peisenieks avatar
Janis Peisenieks

The thing is that I’ve never been able to replicate on my own, and for users it usually starts working if they switch networks from their work to another network (like a hotspot). It seems to be an all-or-nothing for some users. So either all of the things from the CDN load, or none do.

Janis Peisenieks avatar
Janis Peisenieks

Ok, I’ll take a look at that, thanks!

Janis Peisenieks avatar
Janis Peisenieks

I just got a report from a user, and what they see is this. This to me would indicate that something is trying to show something else than what is supposed to be there, since the certificate is issued by AWS, and is working for 99% of our other users.

Janis Peisenieks avatar
Janis Peisenieks

Anyways, thank you for your time and help!

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
Why net::err_cert_authority_invalid error occur?
Basically, there are two main reasons for receiving this kind of error:

The Certificate authority that had issued this certificate is not trusted or the certificate has expired
and *Your system date and time are not correct*.
Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
How To Fix net::err_cert_authority_invalid error (Solved)attachment image

Are you getting net::err_cert_authority_invalid error on Google Chrome? If yes then read this guide to fix this error. As you … Continue Reading

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

Your system date and time are not correct could be the issue

2019-11-26

Rich Allen avatar
Rich Allen

Question here. I’m trying to run the following on a cron like schedule.

docker run --rm -it -u $(id -u):$(id -g) linkchecker/linkchecker <https://somesite.com> -t 40 -o html > ourreport.html
Rich Allen avatar
Rich Allen

Spinning up an aws fargate cluster + vpc and running this on a schedule, seems like way overkill. Is there a better on-demand approach here? Lambda would timeout.

2019-11-27

Janis Peisenieks avatar
Janis Peisenieks
01:34:44 PM

@Andriy Knysh (Cloud Posse) Looks like blacklisting is the issue here. This is from Cisco/Talos’ site:

@Janis Peisenieks the domain name itself should not be suspicious to any enterprise firewalls unless it’s explicitly blocked (blacklisted)

Janis Peisenieks avatar
Janis Peisenieks
Janis Peisenieks avatar
Janis Peisenieks

I guess this means that one of the core problems in my case is that I didn’t know where to look to see if the domain was being blacklisted.

    keyboard_arrow_up