Proxmox VE 8.1.3 - TrueNAS Scale 23.10

[Dennis-Ayala]

Host: Proxmox VE 8.1.3
NAS: TrueNAS Scale 23.10

Target Global Configuration
iqn.2005-10.org.freenas.ctl
15%
3260

Portals:
1
10.xx.xx.xx:3260
Discovery/Auth: None

Initiator Groups:
2
iqn.1993-08.org.debian:01:xxxxxxxxxxx

Targets:

datastore-ssd
Network 10.x.x.x/23
Portal Group ID: 1
Initiator Group ID: 2
Authentication Method: None

In proxmox Server ZFS over ISCSI:

ID: SSD-PROD
Portal: 10.XXX.XXX.XXX
Pool: Actual-Pool-Name
ZFS Block Size: 16k
Target: iqn.2005-10.org.freenas.ctl:datastore-ssd
API Username: root
iSCSI Provider: FreeNAS-API
API IPv4 Host: 10.xx.xx.xx (same as portal)
API Password: password

Results of restoring a VM from backup into the SSD-PROD volume:

new volume ID is 'SSD-PROD:vm-106-disk-0'
iscsiadm: No session found.
restore proxmox backup image: /usr/bin/pbs-restore --repository root@pam@10.XX.XX.XX:BKP-REPO vm/106/2024-01-06T04:54:20Z drive-scsi0.img.fidx iscsi://10.XXX.XXX.XXX/iqn.2005-10.org.freenas.ctl:datastore-ssd/0 --verbose --format raw
connecting to repository 'root@pam@10.XXX.XXX.XXX:BKP-REPO'
open block backend for target 'iscsi://10.XXX.XXX.XXX/iqn.2005-10.org.freenas.ctl:datastore-ssd/0'
iSCSI: Failed to connect to LUN : Failed to log in to target. Status: Target not found(515)
temporary volume 'SSD-PROD:vm-106-disk-0' sucessfuly removed
error before or during data restore, some or all disks were not completely restored. VM 106 state is NOT cleaned up.
TASK ERROR: command '/usr/bin/pbs-restore --repository root@pam@10.XXX.XXX.XXX:BKP-REPO vm/106/2024-01-06T04:54:20Z drive-scsi0.img.fidx iscsi://10.XXX.XXX.XXX/iqn.2005-10.org.freenas.ctl:datastore-ssd/0 --verbose --format raw' failed: exit code 255

Any ideas?

Regards,

Dennis

[Dennis-Ayala]

Ok, so I erased the TrueNAS Scale 23.10.01 (Cobia) and installed TrueNAS Scale 22.02.04 (Angelfish) and was able to get everything working as extecped. There is definitely something to be updated for the plugin to work with Cobia.

Dennis

[matveevandrey]

Does not work with 23.10 iether

[awptechnologies]

mine works im on latest TrueNAS-SCALE-23.10.1.3

[Alcatraz077]

Solution I've found is just to add your pub key into the "Authorized Keys" section of the root user.

To do so:
Copy the contents of your .pub key file generated from the documented command (ssh-keygen -f /etc/pve/priv/zfs/$IP_id_rsa)

In TrueNAS:

• Navigate to Credentails -> Local Users
• Expand the root user
• Click Edit
• Paste the contents of your public key into the Authorized Keys section
• Save

Edit: This was with TrueNAS-SCALE-23.10.1.3 and Proxmox VE 8.1.0
Edit 2: When I say "solution" I mean Proxmox can see and login to TrueNAS as the disabled "root" user by default in TrueNAS 23. If I try to move a disk to TrueNAS, I get the same error that OP reported, even though the disk does show as a new zVol in TrueNAS (just only ~56KB and not anywhere near the actual size of the disk)

[adam-beckett-1999]

I'm getting the same result as Alcatraz077. Connected, and showing correctly within proxmox UI, but iscsiadm discovery shows nothing and when creating a VM it fails with the error: 'TASK ERROR: unable to create VM 200 - Unable to connect to the FreeNAS API service at '192.168.100.1' using the 'http' protocol at /usr/share/perl5/PVE/Storage/LunCmd/FreeNAS.pm line 380.'

Disk does actually get created on the TrueNAS server but it never gets any further than that.

[cgg507]

I do confirm it's working as expected using PVE 8.1.10 + TrueNAS Scale 23.10.2. Took me a while to figure out a couple of things.

I think @adam-beckett-1999 needs to enable "API use SSL" when creating the ZFS over iSCSI connection on PVE end. I can post /etc/scst.conf (Scale-end configuration) and /etc/pve/storage.cfg as needed.

[adam-beckett-1999]

I did try with both SSL enabled and not, and it didn't seem to make any difference. Same errors. If you have a working configuration, please do share.

[cgg507]

PVE 8.1.10 IP address = 192.168.253.251

TrueNAS Scale 23.10.2= 192.168.253.252

I followed these steps:

1. Installed latest freenas-proxmox using the readme, stable release.
2. Thanks to Proxmox VE 8.1.3 - TrueNAS Scale 23.10 #174 (comment) I copied the generated ssh public key into Authorized Keys for the root user in Scale.
3. Used these settings in Scale:

Pool Name:

• Casa

iSCSI Settings:

• Target Global Config / Base Name = iqn.2005-10.org.freenas.ctl
• Portals / Created one with id 1, configured Description and ip address (which is the IP address of the TrueNAS interface) 192.168.253.252.
• Initiator Groups / Created one with id 1, configured with "Add allowed Initiators IQN" and pasted the iSCSI initiator IQN (its on the last line of /etc/iscsi/initiatorname.iscsi in each PVE host). Need to add an entry for each host.
• Targets / Created one entry - target name: scale, Portal Group IP - chose portal id 1 (created earlier), Initiator group id - chose initiator group id 1 which contained my sole host iqn.

To confirm iscsi connectivity is possible, you can test with the following command in PVE shell (replace the IP after the -p option as needed).
This should show the iqn of the target (Scale in this case). If there's no reply then there's some configuration issue on Scale end.
iscsiadm -m discovery -t st -p 192.168.253.252

Example:

---

Current contents of my config files.

PVE host /etc/initiatorname.iscsi

root@casa:~# cat /etc/iscsi/initiatorname.iscsi
InitiatorName=iqn.1993-08.org.debian:01:359bae146e

PVE host /etc/pve/storage.cfg

zfs: Scale
        blocksize 16k
        iscsiprovider freenas
        pool Casa
        portal 192.168.253.252
        target iqn.2005-10.org.freenas.ctl:scale
        content images
        freenas_apiv4_host 192.168.253.252
        freenas_password SuperSecureAndComplicatedPassword.
        freenas_use_ssl 1
        freenas_user admin
        nowritecache 0
        sparse 1

TrueNAS Scale /etc/scst.conf - parts of these are automatically filled by TrueNAS GUI, others were added by creating a VM with id 105 on PVE:

root@truenas[~]# cat /etc/scst.conf 
TARGET_DRIVER copy_manager {
        TARGET copy_manager_tgt {
                LUN 0 Casa-vm-105-disk-0
        }
}

HANDLER vdisk_fileio {
}
HANDLER vdisk_blockio {
    DEVICE Casa-vm-105-disk-0 {
        filename /dev/zvol/Casa/vm-105-disk-0
        blocksize 512
        read_only 0
        usn d3d6bbefcec8f8a
        naa_id 0x6589cfc000000f6d1f88964677a53073
        prod_id "iSCSI Disk"
        rotational 0
        t10_vend_id TrueNAS
        t10_dev_id d3d6bbefcec8f8a 
    }

}

TARGET_DRIVER iscsi {
    enabled 1

    TARGET iqn.2005-10.org.freenas.ctl:scale {
        rel_tgt_id 1
        enabled 1
        per_portal_acl 1

        GROUP security_group {
            INITIATOR iqn.1993-08.org.debian:01:359bae146e\#192.168.253.252

            LUN 0 Casa-vm-105-disk-0
        }
    }
}

kvm process with iscsi session: