Keeping Pith Desk Small with RustDesk and i3

I wanted to use Pith Desk on my home server from another device. Since its interface was already built with web technology, opening it in a browser seemed like a small extension.

The discussion soon included remote authentication, independent client views, file uploads, interactive terminals, browser sessions, and application previews. I was starting to design an entire remote workstation around an agent application.

I ended up running the Linux desktop version of Pith Desk alongside a browser and a terminal, arranging their windows with i3, and accessing the desktop through RustDesk.

The resulting layout is straightforward:

My device
    |
    | RustDesk
    v
Linux desktop on the home server
    |
    +-- i3 window layout -------------------------------+
    |                                                  |
    |  Pith Desk       Chromium          Terminal      |
    |  conversation    websites          shell         |
    |  approvals       app previews      manual checks |
    |                                                  |
    +--------------------------------------------------+

Pith Desk remains a desktop application in this arrangement. The remote desktop supplies the surrounding working environment.

Why browser access looked like a small change

Pith Desk uses the Pith Go library for agent execution and MyGo for its native window. Its web interface communicates with the Go service over HTTP and WebSocket connections. A browser preview mode already exists.

The service also owns the running task. Losing a browser subscription does not, by itself, stop the agent. That was a useful starting point for remote access.

However, the local service was designed around a loopback address and a trusted desktop interface. Making it remotely accessible introduces a different set of questions. Which device receives a copied response? Where does a save dialog appear? When a user selects a workspace, whose filesystem are they browsing?

There was also shared navigation state. In the implementation I started with, the selected conversation belonged to the service. A phone switching to conversation B could change the view shown on the desktop. Locks around task execution would not make those two interfaces independent.

One person using a laptop and a phone is enough to encounter a multiple-client problem.

A chat page grew into a workstation

I considered keeping server and desktop operation separate. Each instance would own its execution environment and data, which would reduce the coordination required between a local desktop and a remote backend.

That clarified where work would happen. It also made the next requirement obvious: if the browser page was my only interface to the server, it needed to support the rest of the work.

Each item has a reasonable implementation. Together, they substantially expand the application.

An agent command log does not give me an interactive shell. A terminal needs its own process, input handling, resizing, and connection lifecycle. A file link needs a defined relationship to the server workspace. A browser used by an agent needs persistent state and a way for a person to take over.

These details become part of the product once users depend on them.

Reusing components still left integration work

I explored reusing a visible Chromium container with Playwright MCP, ttyd with tmux for a terminal, and the existing Go service for file transfer. A remote browser could also open development services inside the server environment.

Embedding the browser viewer and terminal inside Pith Desk was feasible. It would have given me a conversation on one side and a tool panel on the other, without switching to separate pages.

Even with those components, Desk would still need to coordinate authentication, workspace paths, keyboard focus, clipboard behavior, resizing, and connection recovery. Hiding a panel would need to stop unnecessary streaming while preserving the underlying browser or terminal session.

I could reuse the terminal emulator and the remote display software. The integration would remain mine to maintain.

At that point, a desktop became a much more appealing place to compose the tools.

Trying a browser desktop exposed the interaction problem

My GMK homelab already had a remote desktop environment based on LinuxServer Webtop. It provided a Linux desktop through a browser, with a terminal, Chromium, and ordinary file tools available in the same environment.

Running the Linux build of Pith Desk there let me use existing applications for the surrounding work. The agent application could stay focused on conversations, execution, and approvals.

The phone experience was less comfortable. Small window controls, pointer movement, scrolling, and the space consumed by the software keyboard all mattered. A desktop streamed to a narrow screen still has desktop interaction patterns.

This also exposed a useful debugging trap. I initially treated tiny window buttons as an application problem, and the work expanded into Linux window decoration adjustments. I later realized that the remote display resolution had made the controls appear too small. The display configuration deserved attention before adding more application behavior.

The experience made me separate two goals: convenient access to a conversation from a phone, and access to a complete development environment. They benefit from different interfaces.

RustDesk handles access and i3 handles the layout

The setup I settled on uses RustDesk for remote desktop access and i3 to arrange the application windows.

i3 is a tiling window manager. Its role here is specific: keep Pith Desk, Chromium, and a terminal visible together, with predictable placement and a way to give one application more space when needed.

That directly addresses my reluctance to keep opening separate pages. The conversation, browser, and terminal are separate applications, but they occupy one working surface. I can read an agent reply, inspect the running application, and enter a command without leaving the remote environment.

RustDesk carries my display and input. It does not add browser automation tools to Pith. Any agent access to a browser still depends on the configured tools or MCP connection. Keeping that distinction clear helps avoid confusing human remote access with agent computer control.

The desktop also gives a development service a natural place to be viewed: a browser running in the same remote environment. A separate public preview gateway can wait until I need to open that application directly on my own device.

File transfer remains a distinct concern as well. Placing windows next to each other does not move files between machines. Existing Desk downloads and the chosen transfer channel can handle that without turning the application into a general file hosting service.

The maintenance savings have a resource cost

This arrangement keeps a graphical session, a browser, terminal processes, and remote display software running on the server. It still needs persistent application data, browser profiles, working directories, and a reliable startup procedure.

The savings are in the responsibilities I keep inside Pith Desk. I can use the desktop’s window management and existing applications instead of adding a terminal session manager and an embedded browser viewer to the product.

A shared desktop also has shared focus. Two connected devices can interfere with the same pointer, keyboard, and windows. I accept that for a personal workspace where I take turns between devices. Independent simultaneous use would require a different design.

I also keep the phone limitation in perspective. RustDesk gives me another way to operate the desktop; it does not make the applications into native mobile interfaces. Long conversations and quick approvals may still justify a small, purpose-built web interface later.

What I would ask of a future server mode

A future server mode should start with a narrower job. Reading progress, sending a message, and approving an action from a phone could be enough to justify it. Those actions would still need clear client state, authentication, and retry behavior, but they would leave the full desktop workflow where it already works.

For the broader development workflow, I now have Pith Desk, a browser, and a terminal together on the server. i3 supplies the layout, and RustDesk supplies access to it. That lets me spend more of the application’s maintenance budget on agent execution, permissions, history, and recovery.