How to generate a self signed certificate for client and server in OpenSSL 3.0?

[sshedi]

I'm currently doing this:

#!/bin/bash

set -x

cat > v3.ext <<-EOF
authorityKeyIdentifier=keyid,issuer
basicConstraints=CA:FALSE
keyUsage = digitalSignature, nonRepudiation, keyEncipherment, dataEncipherment
extendedKeyUsage = serverAuth
subjectAltName = @alt_names

[alt_names]
IP.1=192.168.1.100
DNS.1=192.168.1.100
EOF

openssl genrsa -out ca.key 4096

openssl req -x509 -new -nodes -sha512 -days 3650 -subj "/C=CN/ST=BLR/L=BLR/O=Org/OU=TEC/CN=192.168.1.100" -key ca.key -out ca.pem -outform PEM

# server.pem
openssl req -sha512 -new -subj "/C=CN/ST=BLR/L=BLR/O=Org1/OU=TEC/CN=192.168.1.100" -key ca.key -out server.csr

openssl x509 -req -sha512 -days 365 -CA ca.pem -CAkey ca.key -CAcreateserial -in server.csr -out server.pem \
  -extfile v3.ext -outform PEM

# client.pem
openssl req -sha512 -new -subj "/C=CN/ST=BLR/L=BLR/O=Org1/OU=TEC/CN=192.168.1.100" -key ca.key -out client.csr

openssl x509 -req -sha512 -in client.csr -CA ca.pem -CAkey ca.key -CAcreateserial -out client.pem -days 365 -outform PEM

Server code:

#!/usr/bin/env python3

import http.server, ssl, socketserver
from http.server import SimpleHTTPRequestHandler, HTTPServer

addr = ("192.168.1.100", 4443)

httpd = HTTPServer(addr, SimpleHTTPRequestHandler)

httpd.socket = ssl.wrap_socket(httpd.socket,
                               certfile="server.pem",
                               server_side=True,
                               keyfile="ca.key")
httpd.serve_forever()

After this, I did:

openssl s_client -connect 192.168.1.100:4443 -showcerts

And:

CONNECTED(00000003)
Can't use SSL_get_servername
depth=0 C = CN, ST = BLR, L = BLR, O = Org1, OU = TEC
verify error:num=20:unable to get local issuer certificate
verify return:1
depth=0 C = CN, ST = BLR, L = BLR, O = Org1, OU = TEC
verify error:num=21:unable to verify the first certificate
verify return:1
depth=0 C = CN, ST = BLR, L = BLR, O = Org1, OU = TEC
verify return:1
---
Certificate chain
 0 s:C = CN, ST = BLR, L = BLR, O = Org1, OU = TEC
   i:C = CN, ST = BLR, L = BLR, O = Org, OU = TEC
...

So I copied ca.pem to /etc/ssl/certs and rehashed pem files.
And after that s_client started working from within the same host.

Then I copied client.pem to a different host and added it to trust store in that machine.
And I did s_client from the other machine, it is showing:

CONNECTED(00000003)
Can't use SSL_get_servername
depth=0 C=CN, ST=BLR, L=BLR, O=Org1, OU=TEC
verify error:num=20:unable to get local issuer certificate
verify return:1
depth=0 C=CN, ST=BLR, L=BLR, O=Org1, OU=TEC
verify error:num=21:unable to verify the first certificate
verify return:1
depth=0 C=CN, ST=BLR, L=BLR, O=Org1, OU=TEC
verify return:1
---
Certificate chain
 0 s:C=CN, ST=BLR, L=BLR, O=Org1, OU=TEC
   i:C=CN, ST=BLR, L=BLR, O=Org, OU=TEC
   a:PKEY: rsaEncryption, 4096 (bit); sigalg: RSA-SHA512
...

What am I doing wrong? Please assist.

[vdukhovni]

You say you want self signed certificates for a client and server, but really you appear to be working to use a private CA to sign certificates for a client and server, and to trust that CA to authenticate connections between them.

Do you expect to deploy more clients and servers that will trust this shared CA? Or this is only ever going to be a single pair, in which case, you may be simpler to bypass the CA, and directly authorise connections from the peer certificate (client or server)?

Is this a lab or development test bed, or a tentative "production" deployment?

In terms of what you're doing wrong:

1. Importing the client rather than the CA certificate into the trust store.
2. Using the same underlying private key for all three of CA, server and client.
3. Possibly setting a persistent CA directory that tracks past issued certificates and next serial number, ...
4. Using IP addresses as DNS names
5. Giving the client and the server the same subject Distinguished Names (DNs)
6. Giving the CA a distinguished name that is too similar to the server's.