Interacting with GUI from CLI (headless GUI)

I have linux server.

I expect there’s headless GUI that can be controlled by CLI for my server.

I do know it possible display GUI with XRDP. But I expect I can control it with SSH or CLI. It works fine when i’m using XRDP.

I have OpenBox (a windows manager) installed. I expect I can interact GUI with CLI or maybe there’s Python Library that can handle it.

mouseclick(2,3) # mouse click area at coordinate (2,3)

screenshot("./current_screen.png") # saving screenshot of current screen in specified path.

And another feature that library can handle it.

I found similiar library, it was pyautogui.

But pyautogui work if there’s existing GUI. I mean the python script error Display Not Found if i run it in CLI.

import pyautogui

It gave me error:

root@server-kentang:~/py# python3
Traceback (most recent call last):                           File "/usr/local/lib/python3.8/dist-packages/Xlib/support/", line 76, in get_socket
    s.connect('/tmp/.X11-unix/X%d' % dno)                  FileNotFoundError: [Errno 2] No such file or directory

During handling of the above exception, another exception occurred:

Traceback (most recent call last):                           File "", line 1, in <module>
    import pyautogui
  File "/usr/local/lib/python3.8/dist-packages/pyautogui/", line 249, in <module>
    import mouseinfo
  File "/usr/local/lib/python3.8/dist-packages/mouseinfo/", line 223, in <module>
    _display = Display(os.environ['DISPLAY'])
  File "/usr/local/lib/python3.8/dist-packages/Xlib/", line 80, in __init__
    self.display = _BaseDisplay(display)
  File "/usr/local/lib/python3.8/dist-packages/Xlib/", line 62, in __init__
    display.Display.__init__(*(self, ) + args, **keys)
  File "/usr/local/lib/python3.8/dist-packages/Xlib/protocol/", line 58, in __init__
    self.socket = connect.get_socket(name, host, displayno)
  File "/usr/local/lib/python3.8/dist-packages/Xlib/support/", line 76, in get_socket
    return mod.get_socket(dname, host, dno)
  File "/usr/local/lib/python3.8/dist-packages/Xlib/support/", line 78, in get_socket
    raise error.DisplayConnectionError(dname, str(val))
Xlib.error.DisplayConnectionError: Can't connect to display ":0": [Errno 2] No such file or directory

It seems that Environment Variable DISPLAY was not set correctly here.

It should be set to something like this:
export DISPLAY=:0.0
Or more generally:
Where $HOSTNAME & $N & $W should match the Existing Situation.

In OP case, HOSTNAME is empty (which means localhost) & N is 10 (which can change over time at each Execution Instance of the X Server) & W is 0 (which may be mostly constant) to get a working configuration.

Answered By: Prem
Categories: Answers Tags: , ,
Answers are sorted by their score. The answer accepted by the question owner as the best is marked with
at the top-right corner.