#bastion

Discuss cloudposse/bastion

2019-10-17

julien M.

Hi, i try to use the cloudposse bastion with docker-compose but i have this error when i try to ssh connect ssh <mailto:[email protected]> -i loadServer -p 1234 [email protected]: Permission denied (publickey).

julien M.

i see this error on bastion container :

bastion_1  \| AuthorizedKeysCommand /usr/bin/github-authorized-keys jmenan failed, status 7
bastion_1  \| AuthorizedKeysCommand /usr/bin/github-authorized-keys jmenan failed, status 7
bastion_1  \| Connection closed by authenticating user jmenan x.x.x.x port 50845 [preauth]
julien M.

this is my bastion.env :

API_URL=<http<i class="em em-//gak"></i>301/user/%s/authorized_keys>
MFA_PROVIDER=google-authenticator
SSH_AUDIT_ENABLED=false
SSH_AUTHORIZED_KEYS_COMMAND=/usr/bin/github-authorized-keys
SSH_AUTHORIZED_KEYS_COMMAND_USER=root
LOGLEVEL=DEBUG
julien M.

and my gak.env

GITHUB_API_TOKEN=xxx
GITHUB_ORGANIZATION=xxx
GITHUB_TEAM=ssh
SYNC_USERS_GID=500
SYNC_USERS_GROUPS=sudo
SYNC_USERS_SHELL=/bin/bash
SYNC_USERS_ROOT=/
SYNC_USERS_INTERVAL=30
ETCD_ENDPOINT=<http<i class="em em-//etcd"></i>2379>
ETCD_TTL=30
ETCD_PREFIX=github-authorized-keys
LISTEN=:301
INTEGRATE_SSH=true
LOG_LEVEL=debug
LINUX_USER_ADD_TPL=adduser -D -s {shell} {username}
LINUX_USER_ADD_WITH_GID_TPL=adduser -D -s {shell} -u {gid} {username}
LINUX_USER_ADD_TO_GROUP_TPL=addgroup {group}
SSH_AUTHORIZED_KEYS_COMMAND_USER=root
SSH_RESTART_TPL=echo "/usr/sbin/service ssh force-reload"

2019-10-15

guigo2k

after working fine for several months, bastion is now showing error {"job"<i class="em em-"syncUsers","level""></i>"error","msg":"Access denied","subsystem"<i class="em em-"jobs","time""></i>"2019-10-15T12<i class="em em-17"></i>08Z"}

guigo2k
cloudposse/bastion

Secure Bastion implemented as Docker Container running Alpine Linux with Google Authenticator & DUO MFA support - cloudposse/bastion

guigo2k

any ideas?

Erik Osterman

Github rate limits?

Erik Osterman

Github token revoked

2019-09-17

Bruce

Hey guys has any integrated okta with the docker bastion?

Erik Osterman

@Bruce haven’t tried…. just duo for MFA

Erik Osterman

(we use teleport with okta)

Bruce

Thanks @Erik Osterman could teleport run on ECS?

Erik Osterman

Hrmm… we haven’t deployed it that way, but I suppose

1
Erik Osterman

there’s a gravitational slack team. they can probably point you in the right direction.

Erik Osterman
Gravitational Community

A place to discuss Teleport and Gravity

2019-08-09

Erik Osterman

I think it is probably something with the latest Pr that added support for compose

Erik Osterman

I don’t have time to look into it, but will promptly review any PR associated with it

1
Blaise Pabon

Do we have any description of bastion running on AWS? I would like to set up bastion as a jumphost to several services planned for EC2 and EKS. I’m not clear on how to set up the networking for the bastion… Do I put it in a security group with inbound / outbound TCP 22 * and then approve its IP address on the inbound rules for the app servers? Also, do I standup a generic t2.micro to run the Docker image, or is there a recommended AMI? BTW, I’m willing to contribute my findings to make it easier for the next guy.

Erik Osterman

no examples of that. we’ve moved on to using gravitational for SSH, so we’re not using this in any engagements.

Erik Osterman
Modern Privileged Access Management | Teleport | Gravitational

Make it easy for users to securely access infrastructure, while meeting the toughest compliance requirements.

Blaise Pabon

Oh, I didn’t know about gravitational, thanks!

2019-08-08

Thomas Sandberg

Hi! I’m trying to get the bastion up running on my ubuntu docker server. Followed the example on git. Edited the bastion.env and gak.env and copied them to the folder on the server containing the dockercomposer.yml Edited the composer-file to enable bastion. But when I start it everything starts OK accept the bastion-container. I get some error messages on the server like this:

Thomas Sandberg
Thomas Sandberg

It seems like the problem is those ssh-scripts.

Thomas Sandberg
Thomas Sandberg

I guess I have forgot to do something else…. any tips?

2019-04-10

Mohamed.Naseer
04:35:57 AM

@Mohamed.Naseer has joined the channel

2019-04-05

oscarsullivan_old
01:32:29 PM

@oscarsullivan_old has joined the channel

oscarsullivan_old

Do you install Bastion on to every machine or just your one jump host that connects to your internal machines on private subnets?

xluffy

bastion is a jump host

oscarsullivan_old

Thanks that’s what I thoght

oscarsullivan_old

here’s an updated readme PR @Erik Osterman https://github.com/cloudposse/bastion/pull/43

Improve README by osulli · Pull Request #43 · cloudposse/bastion

What Improves readme with the following: Fixes missing backslash in example Makes assumptions easier to read Restructures into a much more readable format Makes shell examples easier to copy and p…

Erik Osterman

( I am out of town, will check on Monday! Thanks @oscarsullivan_old )

joshmyers
09:23:47 PM

@joshmyers has joined the channel

2019-04-03

roco
06:46:46 PM

@roco has joined the channel

2019-04-02

Paul Calabro
07:43:00 PM

@Paul Calabro has joined the channel

Paul Calabro

so erik, what are your thoughts on that?

Paul Calabro

(from the other channel)

Erik Osterman

So I thought of somewhat interesting way that a bad actor could bypass 2FA and I’m curious if this of interest to anyone else.

Scenario:

A bad actor compromises a machine in an untrusted network and quietly imposes these SSH settings on a user (or maybe they’re already in place b/c the user put them there for a reason– e.g. Ansible):

Host *
  ControlMaster auto
  ControlPath ~/.ssh/control-sockets/%C
  ControlPersist yes
  ServerAliveCountMax 5
  ServerAliveInterval 60

…and then they wait for a user to connect to a host behind a bastion using 2FA. Once they do, the bad actor can then reuse that socket over and over again unbeknownst to the user to create sessions using that established connection. And then, course, pivot from there.

AFAIK, unless the bastion server modifies the MaxSessions value, the default number of sessions is 10.

What are your thoughts on this?

1
Erik Osterman

So this would be a problem if the person’s workstation were compromised, no?

Paul Calabro

correct

Paul Calabro

with those kinds of settings in place, 2FA is only prompted on the first attempt

Paul Calabro

i was trying to think of an fun analogy and the best i could come up with is someone using a door wedge, haha

Erik Osterman

True it is like a door wedge

Erik Osterman

But the same thing exists with ssh agents

Erik Osterman

And sockets

Erik Osterman

I am not averse to adding support to optionally disabling these settings

Erik Osterman

Not sure what the default should be

Erik Osterman

Often times, if the bad actor has control of the workstation all bets are off

Paul Calabro

yeah, those are all good points. i think the bastion is a little unique though in that you’re not just using your keys you’ve already added to the ssh agent. you’re also using push notification/sms/etc as that second factor…however, those gets bypass completely in this scenario.

Erik Osterman

Yea true

Erik Osterman

What if we had some SSH server config profiles? E.g. configs with an extension

Erik Osterman

When starting the container we install one of those config profiles

Paul Calabro

yeah, i was thinking of that as well. i use configs in my docker compose file to mount files.

Erik Osterman

One profile could be the one you suggest and maybe we make that default

Paul Calabro

that works

Erik Osterman

If you want to open a PR for that we will promptly review

Paul Calabro

i came across this accidentally and just thought i’d share b/c it seems like an interesting way to misuse an ssh feature

Paul Calabro

sweet! thanks!

Erik Osterman

Yea makes sense

Erik Osterman

It’s a very popular repo and #1 search result

Erik Osterman

So we should make it as locked down as possible

Paul Calabro

yeah, i’m a fan of this project

Erik Osterman

Very cool!

Paul Calabro

it’s good stuff!

Erik Osterman

If you have a chance to audit our scripts in that project, that would be appreciated as well

Paul Calabro

yeah, i’d be happy to take a look

1
Bruce
10:59:23 PM

@Bruce has joined the channel

12:04:16 AM

@ has joined the channel

Vidhi Virmani
06:01:10 AM

@Vidhi Virmani has joined the channel

kskewes
06:28:42 AM

@kskewes has joined the channel

2019-03-20

Tim Malone
11:11:48 PM

@Tim Malone has joined the channel

2019-03-19

xluffy
03:37:44 PM

@xluffy has joined the channel

2019-03-18

Mike Nock
06:53:26 PM

@Mike Nock has joined the channel

2019-03-15

Leo Starcevic
10:27:10 AM

@Leo Starcevic has joined the channel

Leo Starcevic

Hey guys!! Awesome work with the bastion container, but I have one issue, when building the image myself it won’t run, I get

Initializing ssh-audit
- Enabling SSH Audit Logs
Password: chsh: PAM: Authentication token manipulation error
FATAL: Failed to initializeInitializing ssh-audit
- Enabling SSH Audit Logs
Password: chsh: PAM: Authentication token manipulation error
FATAL: Failed to initialize

Using the image from docker hub works fine though, any idea what could be wrong?

Erik Osterman

Hrmmm… not sure off the top of my head

Erik Osterman

Seems to be passwd related which is why the chsh error is emitted

Erik Osterman

Where are you running the container?

Leo Starcevic

aws, but I tried locally on my ubuntu machine as well, same error

Erik Osterman

How are you invoking it?

Erik Osterman

When you use our version, how are you referencing the image? With a release tag or latest?

Leo Starcevic
docker run -p 1234:22 -d --name bastion \
     -e MFA_PROVIDER=google-authenticator \
     -v ~/.ssh/authorized_keys:/root/.ssh/authorized_keys \
     bastion
Leo Starcevic

bastion is the one I built, cloudposse/bastion works fine though

Erik Osterman

Can you confirm it still works if you pin to the latest release?

Leo Starcevic

what do you mean? that is the latest?!

Erik Osterman

My concern is “latest” might be stale or something… maybe Travis didn’t tag latest

Leo Starcevic

ah ok, so cloudposse/bastion:0.4.4-228 ?

Erik Osterman

Though I think without the -228

Erik Osterman

It’s been a while since I looked at Travis for that project

Leo Starcevic

0.4.4 from docker hub works fine as well

Erik Osterman

Ok very odd indeed

Erik Osterman

I am afk, but will test when I get to the office

Leo Starcevic

could it be something gets a later version when I run the docker build today, 0.4.4 was built like a month ago

Leo Starcevic

thanks, no rush!

Erik Osterman

Possibly - we try to pin most things down to a version

Erik Osterman

But not down to the package level

2019-03-11

Juan Cruz Diaz
03:10:25 PM

@Juan Cruz Diaz has joined the channel

2019-03-06

11:25:29 AM

@ has joined the channel

2019-03-04

Erik Osterman
04:05:28 PM

@Erik Osterman has joined the channel

Erik Osterman
04:05:28 PM

@Erik Osterman set the channel purpose: Discuss cloudposse/bastion

hairyhenderson
04:05:35 PM

@hairyhenderson has joined the channel

hairyhenderson

w00t

Erik Osterman

What’s your challenge?

hairyhenderson

so I’m having a lot less trouble than I was

hairyhenderson

the helm chart is… rusty

Erik Osterman

Haha yes

Erik Osterman

We are using teleport with our customers now

hairyhenderson

oh really!

Erik Osterman

But we maybe able to address/fix problems in bastion

hairyhenderson

I’d looked at that, but bastion seems so much simpler

aknysh
04:07:44 PM

@aknysh has joined the channel

Erik Osterman

Yes teleport is much more complicated, but also more feature rich

Erik Osterman

Bastion is quite simple by comparison

Erik Osterman

How far did you get and what’s the current problem?

hairyhenderson

sec - got pulled into a call

Erik Osterman

No worries! Just post back here when you are free. I will also be jumping on a call shortly.

hairyhenderson

ok so… the initial problem was I was constantly getting Connection closed by authenticating user hairyhenderson 127.0.0.1 port 49698 [preauth] errors, and I couldn’t see github-authorized-keys being called, but it turns out the commented-out env vars in the values.yaml aren’t all the default values - specifically SSH_AUTHORIZED_KEYS_COMMAND: "/usr/bin/github-authorized-keys" is super-important to uncomment

hairyhenderson

I’ll probably issue a PR with a bunch of updates once I’m through the learning curve

Erik Osterman

Certainly - we’ll get that approved quickly

Erik Osterman

ping me here, if you’re blocked

Erik Osterman

the helm chart was a bit complicated as I recall due to the need for initializing a shared volume in /etc/ and running github-authorized-keys as a sidecar

Erik Osterman
05:12:55 PM

@Erik Osterman set the channel topic: https://github.com/cloudposse/bastion

05:30:41 PM

@ has joined the channel

tamsky
09:58:32 PM

@tamsky has joined the channel

    keyboard_arrow_up