Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

local error: tls: bad record MAC #2838

Closed
jceloria opened this issue Jul 2, 2020 · 11 comments
Closed

local error: tls: bad record MAC #2838

jceloria opened this issue Jul 2, 2020 · 11 comments

Comments

@jceloria
Copy link

jceloria commented Jul 2, 2020 •
edited

Summary

This is a new install using docker-compose, following the getting started guide. After running docker-compose up, I proceed to the console and see Token exchange refused. I came across #2353, #1818, #2511, and #2521 all of which led me to try different options to resolve this issue, unfortunately nothing has worked for me thus far.
...

Steps to Reproduce

  1. Configure docker-compose.yml and ttn-lw-stack.yml
  2. Initialize the database, create admin user, create oauth-client using the same value as client-secret as outlined in the getting-started documentation for console.oauth.client-secret
  3. run: docker-compose up

What do you see now?

I'm able to resolve the host name from the container:

/ # ping -q -c1 lora.<redacted>
PING lora.<redacted> (192.168.1.10): 56 data bytes

--- lora.<redacted> ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 0.052/0.052/0.052 ms

I have verified the certificates:

/ # ttn-lw-stack config | grep \.pem | sed 's/^ *//g'
--tls.certificate="/run/secrets/cert.pem"
--tls.key="/run/secrets/key.pem"
--tls.root-ca="/run/secrets/ca.pem"

/ # apk add openssl
fetch http://dl-cdn.alpinelinux.org/alpine/v3.10/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.10/community/x86_64/APKINDEX.tar.gz
(1/1) Installing openssl (1.1.1g-r0)
Executing busybox-1.30.1-r3.trigger
OK: 8 MiB in 19 packages

/ # openssl x509 -in /run/secrets/ca.pem -text -noout | awk '/Subject Key Identifier/ && $0 != "" {getline; print $0}' | tr -d ' '
A8:4A:6A:63:04:7D:DD:BA:E6:D1:39:B7:A6:45:65:EF:F3:A8:EC:A1

/ # openssl x509 -in /run/secrets/cert.pem -text -noout | awk '/Authority Key Identifier/ && $0 != "" {getline; print $0}' | tr -d ' '
keyid:A8:4A:6A:63:04:7D:DD:BA:E6:D1:39:B7:A6:45:65:EF:F3:A8:EC:A1

When I run curl against localhost and the host name I'm using in the config I get the following:

/ # for host in localhost lora.<redacted>; do echo ${host}:; curl https://${host}:8885; echo -e '-----\n'; done
localhost:
curl: (60) SSL certificate problem: unable to get local issuer certificate
More details here: https://curl.haxx.se/docs/sslcerts.html

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.
-----

lora.<redacted>:
curl: (60) SSL certificate problem: unable to get local issuer certificate
More details here: https://curl.haxx.se/docs/sslcerts.html

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.
-----

I get the same outcome when I specify the ca cert:
curl --cacert /run/secrets/ca.pem https://${host}:8885

When viewing the docker logs, I see the following when attempting to connect:

2020/07/02 18:05:42 http: TLS handshake error from 127.0.0.1:47152: local error: tls: bad record MAC
2020/07/02 18:05:42 http: TLS handshake error from 192.168.176.1:36420: local error: tls: bad record MAC

Am I missing something?

What do you want to see instead?

I would like to login to the console.

Environment

#─► rpm -q centos-release docker-ce
centos-release-8.1-1.1911.0.9.el8.x86_64
docker-ce-19.03.10-3.el7.x86_64

/ # ttn-lw-stack version
The Things Stack for LoRaWAN: ttn-lw-stack
Version:             3.8.4
Build date:          2020-06-12T17:15:05Z
Git commit:          d63f7de74
Go version:          go1.14.4
OS/Arch:             linux/amd64

How do you propose to implement this?

no idea.

How do you propose to test this?

n/a

Can you do this yourself and submit a Pull Request?

n/a
@KrishnaIyer
Copy link
Member

  1. How did you generate the SSL Certs? If you've deployed this on a public machine, you can either use the in-built stack ACME or query certificates yourself using Lets Encrypt.
  2. Does the CN (Canonical Name) of the certificate match the host where this is deployed?

If you're using the stack locally (localhost), then I'd recommend interacting with the stack without TLS (using the http/mqtt/ws/grpc ports).

for host in localhost lora.; do echo ${host}:; curl https://${host}:8885; echo -e '-----\n'; done

This is quite wrong. No Certificate Authority will provide you a certificate for localhost and if you're using self-signed certs, please don't use this on a public deployment.

@jceloria
Copy link
Author

jceloria commented Jul 5, 2020 via email •
edited

@benolayinka
Copy link
Contributor

Let me see if understand right

  • You have a machine exposing lora.redacted with Apache using acme certificates for TLS
  • You're running a docker container on this machine but you're not exposing The Things Stack with Apache
  • You're running all of the above commands on this machine, so a ping to localhost will never leave the machine but a ping to lora.redacted should hit a DNS resolver somewhere and then come back to the machine through Apache

If I have that right, is this machine physically local to you? Are you opening a browser on this machine and hitting localhost:1885?

@jceloria
Copy link
Author

jceloria commented Jul 6, 2020 via email

@KrishnaIyer
Copy link
Member

KrishnaIyer commented Jul 6, 2020 •
edited

@jceloria: This has to do with your console/oauth configuration. Can you add that here?

@jceloria
Copy link
Author

jceloria commented Jul 6, 2020

That's most likely the case, I'm sure.

I posted both the docker-compose.yml and ttn-lw-stack.yml, under Steps to reproduce (1.) as gists, or were you referring to something else?

@KrishnaIyer
Copy link
Member

Ah yeah indeed. I just saw that. I think I see the issue here.
For your configuration, only https://lora.redacted will work. Also your config is missing :8885 unless you're proxying https://lora.redacted to https://lora.redacted:8885.

What values did you use as callback when creating the Oauth client?

@jceloria
Copy link
Author

jceloria commented Jul 6, 2020 •
edited

I used Ansible to deploy with the following arguments to docker-compose which I grabbed from the getting-started/installation page:

      - >-
        pull
      - >-
        run --rm stack is-db init
      - >-
        run --rm stack is-db create-admin-user --id admin --email '{{ admin_email }}'
        --password '{{ admin_password }}'
      - >-
        run --rm stack is-db create-oauth-client --id cli --name 'Command Line Interface'
        --owner admin --no-secret --redirect-uri 'local-callback' --redirect-uri 'code'
      - >-
        run --rm stack is-db create-oauth-client --id console --owner admin
        --secret '{{ client_secret }}' --redirect-uri 'https://{{ fqdn }}/console/oauth/callback'
        --redirect-uri '/console/oauth/callback' --logout-redirect-uri 'https://{{ fqdn }}/console'
        --logout-redirect-uri '/console'

{{ fqdn }}: lora.redacted
{{ client_secret }}: is the same value that's in ttn-lw-stack.yml

In all likelihood, I'm missing something that's not obvious to me with the configuration. I appreciate the assistance.

@jceloria
Copy link
Author

jceloria commented Jul 6, 2020

I see what you're saying, since this is all local, I should be able to update the config to the following:

# Web UI configuration
console:
  ui:
    canonical-url: 'https://{{ fqdn }}:1885/console'
    is:
      base-url: 'https://{{ fqdn }}:1885/api/v3'
    gs:
      base-url: 'https://{{ fqdn }}:1885/api/v3'
    ns:
      base-url: 'https://{{ fqdn }}:1885/api/v3'
    as:
      base-url: 'https://{{ fqdn }}:1885/api/v3'
    js:
      base-url: 'https://{{ fqdn }}:1885/api/v3'
    qrg:
      base-url: 'https://{{ fqdn }}:1885/api/v3'
    edtc:
      base-url: 'https://{{ fqdn }}:1885/api/v3'

  oauth:
    authorize-url: 'https://{{ fqdn }}:1885/oauth/authorize'
    token-url: 'https://{{ fqdn }}:1885/oauth/token'
    client-id: 'console'
    client-secret: '{{ client_secret }}'# Web UI configuration
console:
  ui:
    canonical-url: 'http://{{ fqdn }}:1885/console'
    is:
      base-url: 'http://{{ fqdn }}:1885/api/v3'
    gs:
      base-url: 'http://{{ fqdn }}:1885/api/v3'
    ns:
      base-url: 'http://{{ fqdn }}:1885/api/v3'
    as:
      base-url: 'http://{{ fqdn }}:1885/api/v3'
    js:
      base-url: 'http://{{ fqdn }}:1885/api/v3'
    qrg:
      base-url: 'http://{{ fqdn }}:1885/api/v3'
    edtc:
      base-url: 'http://{{ fqdn }}:1885/api/v3'

  oauth:
    authorize-url: 'http://{{ fqdn }}:1885/oauth/authorize'
    token-url: 'http://{{ fqdn }}:1885/oauth/token'
    client-id: 'console'
    client-secret: '{{ client_secret }}'

@KrishnaIyer
Copy link
Member

Exactly. The above looks correct.

@jceloria
Copy link
Author

jceloria commented Jul 6, 2020

Thank you, that was in fact what I needed. After you had mentioned it, I came across #1752. If anyone is curious, this is the scrubbed & stripped down Ansible role that I'm testing with: https://github.com/jceloria/ansible-ttn-stack

Thanks again for everyone's help.

@jceloria jceloria closed this as completed Jul 6, 2020
@benolayinka benolayinka mentioned this issue Jul 7, 2020
Closed
@TheIronNinja TheIronNinja mentioned this issue Sep 23, 2020
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@jceloria @benolayinka @KrishnaIyer