rdp from WIndows 11 to Ubutnu 23.04 authentication error

Trying to configure Ubuntu 23.04 so that I can remote login and manage my Ubuntu desktop from a Windows 11 PC using RDP.

I am getting authentication errors when I try to login from windows via RDP.

I am 100% certain I am typing the password correctly. I have verified the account works properly by logging into it directly.

The user is not logged in on any device.

$ tail /var/log/xrdp-sesman.log
[20230814-14:51:50] [INFO ] starting xrdp-sesman with pid 1243
[20230814-14:53:40] [INFO ] Socket 12: AF_INET6 connection received from ::1 port 34176
[20230814-14:53:42] [ERROR] pam_authenticate failed: Authentication failure
[20230814-14:53:42] [INFO ] AUTHFAIL: user=KenRDP ip=::ffff:192.168.1.12 time=1692042822
[20230814-14:53:42] [ERROR] sesman_data_in: scp_process_msg failed
[20230814-14:53:42] [ERROR] sesman_main_loop: trans_check_wait_objs failed, removing trans


$ cat /var/log/xrdp.log
[20230814-14:51:50] [INFO ] address [0.0.0.0] port [3389] mode 1
[20230814-14:51:50] [INFO ] listening to port 3389 on 0.0.0.0
[20230814-14:51:50] [INFO ] xrdp_listen_pp done
[20230814-14:51:52] [INFO ] starting xrdp with pid 1371
[20230814-14:51:52] [INFO ] address [0.0.0.0] port [3389] mode 1
[20230814-14:51:52] [INFO ] listening to port 3389 on 0.0.0.0
[20230814-14:51:52] [INFO ] xrdp_listen_pp done
[20230814-14:52:48] [INFO ] Socket 12: AF_INET6 connection received from ::ffff:192.168.1.12 port 50168
[20230814-14:52:48] [INFO ] Using default X.509 certificate: /etc/xrdp/cert.pem
[20230814-14:52:48] [INFO ] Using default X.509 key file: /etc/xrdp/key.pem
[20230814-14:52:48] [INFO ] Security protocol: configured [SSL|RDP], requested [SSL|HYBRID|HYBRID_EX|RDP], selected [SSL]
[20230814-14:52:48] [ERROR] SSL_read: I/O error
[20230814-14:52:48] [ERROR] libxrdp_force_read: header read error
[20230814-14:52:48] [ERROR] Processing [ITU-T T.125] Connect-Initial failed
[20230814-14:52:48] [ERROR] [MCS Connection Sequence] receive connection request failed
[20230814-14:52:48] [ERROR] xrdp_sec_incoming: xrdp_mcs_incoming failed
[20230814-14:52:48] [ERROR] xrdp_rdp_incoming: xrdp_sec_incoming failed
[20230814-14:52:48] [ERROR] xrdp_process_main_loop: libxrdp_process_incoming failed
[20230814-14:52:48] [ERROR] xrdp_iso_send: trans_write_copy_s failed
[20230814-14:52:48] [ERROR] Sending [ITU T.125] DisconnectProviderUltimatum failed
[20230814-14:53:19] [INFO ] Socket 12: AF_INET6 connection received from ::ffff:192.168.1.12 port 50170
[20230814-14:53:19] [INFO ] Using default X.509 certificate: /etc/xrdp/cert.pem
[20230814-14:53:19] [INFO ] Using default X.509 key file: /etc/xrdp/key.pem
[20230814-14:53:19] [INFO ] Security protocol: configured [SSL|RDP], requested [SSL|HYBRID|HYBRID_EX|RDP], selected [SSL]
[20230814-14:53:19] [INFO ] Connected client computer name: KEN-X1-CARBON
[20230814-14:53:19] [WARN ] Received [MS-RDPBCGR] TS_UD_HEADER type 0xc006 is unknown (ignored)
[20230814-14:53:19] [WARN ] Received [MS-RDPBCGR] TS_UD_HEADER type 0xc00a is unknown (ignored)
[20230814-14:53:19] [INFO ] xrdp_load_keyboard_layout: Keyboard information sent by the RDP client, keyboard_type:[0x07], keyboard_subtype:[0x00], keylayout:[0x00000409]
[20230814-14:53:19] [INFO ] xrdp_load_keyboard_layout: model [] variant [] layout [us] options []
[20230814-14:53:19] [INFO ] TLS connection established from ::ffff:192.168.1.12 port 50170: TLSv1.3 with cipher TLS_AES_256_GCM_SHA384
[20230814-14:53:19] [INFO ] xrdp_caps_process_pointer: client supports new(color) cursor
[20230814-14:53:19] [INFO ] xrdp_process_offscreen_bmpcache: support level 1 cache size 10485760 MB cache entries 100
[20230814-14:53:19] [INFO ] xrdp_caps_process_codecs: nscodec, codec id 1, properties len 3
[20230814-14:53:19] [WARN ] xrdp_caps_process_codecs: unknown codec id 5
[20230814-14:53:19] [INFO ] Loading keymap file /etc/xrdp/km-00000409.ini
[20230814-14:53:19] [WARN ] local keymap file for 0x00000409 found and doesn't match built in keymap, using local keymap file
[20230814-14:53:40] [INFO ] connecting to sesman on 127.0.0.1:3350
[20230814-14:53:40] [INFO ] xrdp_wm_log_msg: sesman connect ok
[20230814-14:53:40] [INFO ] sesman connect ok
[20230814-14:53:40] [INFO ] sending login info to session manager. Please wait...
[20230814-14:53:42] [INFO ] xrdp_wm_log_msg: login failed for user KenRDP
[20230814-14:53:42] [INFO ] login failed for user KenRDP



sudo systemctl status xrdp
[sudo] password for ken: 
● xrdp.service - xrdp daemon
     Loaded: loaded (/lib/systemd/system/xrdp.service; enabled; preset: enabled)
     Active: active (running) since Mon 2023-08-14 14:51:51 CDT; 50min ago
       Docs: man:xrdp(8)
             man:xrdp.ini(5)
    Process: 1245 ExecStartPre=/bin/sh /usr/share/xrdp/socksetup (code=exited, status=0/SUCCESS)
    Process: 1277 ExecStart=/usr/sbin/xrdp $XRDP_OPTIONS (code=exited, status=0/SUCCESS)
   Main PID: 1371 (xrdp)
      Tasks: 2 (limit: 76779)
     Memory: 42.1M
        CPU: 88ms
     CGroup: /system.slice/xrdp.service
             ├─1371 /usr/sbin/xrdp
             └─4796 /usr/sbin/xrdp

Aug 14 14:53:19 kc-mediaserver xrdp[4796]: [INFO ] xrdp_caps_process_codecs: nscodec, codec id 1, properties len 3
Aug 14 14:53:19 kc-mediaserver xrdp[4796]: [WARN ] xrdp_caps_process_codecs: unknown codec id 5
Aug 14 14:53:19 kc-mediaserver xrdp[4796]: [INFO ] Loading keymap file /etc/xrdp/km-00000409.ini
Aug 14 14:53:19 kc-mediaserver xrdp[4796]: [WARN ] local keymap file for 0x00000409 found and doesn't match built in keymap, using local keymap file
Aug 14 14:53:40 kc-mediaserver xrdp[4796]: [INFO ] connecting to sesman on 127.0.0.1:3350
Aug 14 14:53:40 kc-mediaserver xrdp[4796]: [INFO ] xrdp_wm_log_msg: sesman connect ok
Aug 14 14:53:40 kc-mediaserver xrdp[4796]: [INFO ] sesman connect ok
Aug 14 14:53:40 kc-mediaserver xrdp[4796]: [INFO ] sending login info to session manager. Please wait...
Aug 14 14:53:42 kc-mediaserver xrdp[4796]: [INFO ] xrdp_wm_log_msg: login failed for user KenRDP
Aug 14 14:53:42 kc-mediaserver xrdp[4796]: [INFO ] login failed for user KenRDP

Other troubleshooting I have done that removed previous errors:

$ sudo adduser xrdp ssl-cert
$ sudo systemctl stop xrdp
$ sudo systemctl start xrdp
$ sudo reboot  (rebooted after last change which was to add ssl-cert to xrdp)

Any one know how to resolve the authentication failure? Is there a different approach I should use? My intended use is to manage the Ubuntu PC via a Windows PC within the same LAN (PCs are on different sub-nets).

Asked By: SkiBum

||

The issue was that ssh was not configured on the system. I had only used the system as a local user and never installed/configured/enabled ssh. Once SSH was working properly then RDP worked as expected.

Posting the answer here in case someone else runs into the issue.

Answered By: SkiBum
Categories: Answers Tags: , ,
Answers are sorted by their score. The answer accepted by the question owner as the best is marked with
at the top-right corner.