Skip to content

Access the desktop environment of a workspace #2106

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
Tracked by #5051
kylecarbs opened this issue Jun 6, 2022 · 25 comments
Closed
Tracked by #5051

Access the desktop environment of a workspace #2106

kylecarbs opened this issue Jun 6, 2022 · 25 comments
Labels
api Area: HTTP API enterprise Enterprise-license / premium functionality networking Area: networking site Area: frontend dashboard

Comments

@kylecarbs
Copy link
Member

kylecarbs commented Jun 6, 2022

It'd be useful to access the desktop environment of a workspace in Coder.

  • Cross-platform browser testing
  • Installing applications on Windows (it's hard to install packages on the Windows command line)
  • Desktop application development
  • Android and iOS development
  • ... and I'm sure many more!

This would be implemented as a native access application, similar to the web terminal. A command would be added too, something like: coder desktop <workspace> or coder open desktop <workspace>.

image

Implementation

VNC clients will use noVNC through the browser. This should use noVNC APIs to hide the overlay user interface, making it feel like a native experience.

RDP clients don't currently have a web-based solution (💡 ideas are welcome here). An initial implementation could exclude Windows and still provide significant value, so this shouldn't be a hard blocker!

Although implementations with RDP and VNC are different, we should make this transparent to the user to deliver a consistent experience.

Linux and macOS

Specify a VNC server address in the coder_agent resource (naming subject to change):

resource "coder_agent" "dev" {
  vnc = "tcp://localhost:5900"
}

In the absence of a VNC server specified:

  • On Linux, look for x11vnc on $PATH and start it with the agent when VNC is requested.
  • On macOS, start the VNC server.

Expose the VNC connection through a new DataChannel protocol, and pipe it to a WebSocket endpoint.

Windows

Specify a RDP server address in the coder_agent resource (naming subject to change):

resource "coder_agent" "dev" {
  rdp = "tcp://localhost:3389"
}

In the absence of an RDP server specified, attempt to start it (I googled around a bit, but was unable to find much).

Expose the RDP connection through a new DataChannel protocol, and pipe it to a WebSocket endpoint.

@kylecarbs kylecarbs added site Area: frontend dashboard api Area: HTTP API labels Jun 6, 2022
@f0ssel
Copy link
Contributor

f0ssel commented Jun 6, 2022

I did a quick search for a native go vnc server and didn't find much, which is a bummer because that would fit in really nicely here.

I did find this, but I can't really read it. It has tags for vnc and rdp and is half go half js, so could be something - https://github.com/dushixiang/next-terminal

@bpmct
Copy link
Member

bpmct commented Jun 6, 2022

I've heard great things about apache guacamole: https://guacamole.apache.org/

@bpmct bpmct pinned this issue Jun 6, 2022
@kylecarbs
Copy link
Member Author

Even if there was a native Go VNC server, we'd still need an X11 server so it wouldn't save us much @f0ssel.

We could totally do something with the Apache Guacamole approach. That would solve our Windows woes.

@nikhil96widhani
Copy link

I am not a fan of novnc because there is no shared clipboard and many more things. I absolutely love this image from linuxserver's team https://docs.linuxserver.io/images/docker-webtop

If we can adopt the technology used behind this or simply use this inside a workspace

@AbhineetJain AbhineetJain unpinned this issue Jun 21, 2022
@Emyrk Emyrk pinned this issue Jun 21, 2022
@sreya
Copy link
Collaborator

sreya commented Jun 21, 2022

Kyle and I talked offline about this but this seems better defined as an app than a coder-provided function. Everything that we offer right now works out of the box. For VNC to work correctly the user needs to have already installed x11.

@tjcran
Copy link

tjcran commented Jun 22, 2022

Kyle and I talked offline about this but this seems better defined as an app than a coder-provided function. Everything that we offer right now works out of the box. For VNC to work correctly the user needs to have already installed x11.

This makes sense and I think it aligns well with our proposed templates strategy

@ketang ketang added the enterprise Enterprise-license / premium functionality label Jun 30, 2022
@ntimo
Copy link
Contributor

ntimo commented Jul 2, 2022

I am using https://kasmweb.com/ for desktop apps and it’s truly amazing. Just to drop that there might be useful :)

@ketang ketang unpinned this issue Jul 20, 2022
@coder coder deleted a comment from ketang Aug 24, 2022
@coder coder deleted a comment from tjcran Aug 24, 2022
@coder coder deleted a comment from ketang Aug 24, 2022
@coder coder deleted a comment from ketang Aug 24, 2022
@github-actions
Copy link

This issue is becoming stale. In order to keep the tracker readable and actionable, I'm going close to this issue in 7 days if there isn't more activity.

@github-actions github-actions bot added the stale This issue is like stale bread. label Oct 24, 2022
@kylecarbs kylecarbs removed the stale This issue is like stale bread. label Oct 24, 2022
@ghuntley
Copy link
Contributor

ghuntley commented Nov 3, 2022

@ghuntley ghuntley mentioned this issue Nov 14, 2022
10 tasks
@tidux
Copy link

tidux commented Dec 7, 2022

For Linux desktops, another solution would be using X2go Server, which basically bundles X11 and a few helper protocols over an SSH tunnel. The x2goserver package has been in Debian main and Ubuntu universe package repositories for years now, and since it's just X11 it's DE agnostic. I've used it to great success with Fluxbox on personal machines. The downside is there's no client for iOS or Android or the web.

@phorcys420
Copy link
Member

I like the approach where the template owner defines their own vnc path, it's gotta save some time rather than struggling writing a vnc implementation for the coder agent.
I think we shouldn't hide the overlay though, it allows copy/paste and other resolution fixes when needed

@MrPeacockNLB
Copy link
Contributor

I'm using kasmvnc in my Desktop templates for our eclipse users. It works very well and uses modern technologie transport stream. Even playing youtube videos with low bandwidth is possible.

For me it is on the owner to choose which implementation is used. As we are starting the desktopn environment in the startup script it is more important to give user feedback when one or more services are ready to use. When you think of starting a lot of services this could take some time. Espacially when you have a slow internet connection.

The feedback should not coupled to any service directly. It has to be defined by the owner how many and which services are starting.

@tidux
Copy link

tidux commented Dec 21, 2022

Another thing to keep in mind is that remote desktop protocols are much more latency sensitive than browser based IDEs. Maybe coder open desktop could ICMP ping the target IP to check latency before connecting, and throw a warning if round trip latency is over 50ms?

$ coder open desktop <workspace>
Pinging <workspace> [IPADDR] with 32 bytes of data:
Reply from IPADDR: bytes=32 time=93ms TTL=49
Reply from IPADDR: bytes=32 time=174ms TTL=49
Reply from IPADDR: bytes=32 time=97ms TTL=49
Reply from IPADDR: bytes=32 time=179ms TTL=49
WARNING: latency to Coder workspace <workspace> is over 50ms.  Using a browser IDE or an SSH connection to a local IDE may be preferable!

@phorcys420
Copy link
Member

phorcys420 commented Dec 22, 2022

Another thing to keep in mind is that remote desktop protocols are much more latency sensitive than browser based IDEs. Maybe coder open desktop could ICMP ping the target IP to check latency before connecting, and throw a warning if round trip latency is over 50ms?

I think that if your latency is high you will notice it very quickly.
I think that a separate coder ping command might be useful for debugging though.

@github-actions
Copy link

This issue is becoming stale. In order to keep the tracker readable and actionable, I'm going close to this issue in 7 days if there isn't more activity.

@github-actions github-actions bot added the stale This issue is like stale bread. label Mar 23, 2023
@phorcys420
Copy link
Member

phorcys420 commented Mar 23, 2023

I have a template that uses noVNC+Websockify+TigerVNC to allow for remote desktop: basic-env, you can take a look at it if you want to implement VNC in your Linux template.

As I mentioned in #6448, The Windows implementation doesn't have to use RDP at first, there are VNC servers for windows, which might be a first step towards cross-platform desktop access.

As for Guacamole, there is a "noauth" option, which could make the UX snappier without having to enter a password.
Since Guacamole is multi-protocol, I think it should be the prioritized way to bring a unified UX.

I am using https://kasmweb.com/ for desktop apps and it’s truly amazing. Just to drop that there might be useful :)

The Coder Team is conducting tests on using KasmVNC in templates, and it works atleast as good as my NoVNC template.
I am a bit far away from the servers though, so latency isn't really comparable with my homeserver.
I don't like Kasm's UI much, but that's fine.

@kylecarbs kylecarbs removed the stale This issue is like stale bread. label Mar 23, 2023
@mattlqx
Copy link

mattlqx commented Apr 3, 2023

I have RDP setup via Guacamole and an external URL linked to a button. But I was also looking at what it would take to launch the native Remote Desktop app via button. Looks like the rdp:// or remotedesktop:// schemes are dead (and won't have worked on anything but Windows anyway). It seems like it might be possible to do with some javascript to generate an .rdp file with the right connection options.

Or would using the Coder VSCode extension to proxy open an .rdp file that way be possible?

@matifali
Copy link
Member

We now have a community template that supports Windows RDP in browser OOB. See https://coder.com/docs/v2/latest/ides/remote-desktops#rdp-desktop
related: #6448

@matifali
Copy link
Member

I suppose it would be possible to support RDP OOB for Windows workspaces using the approach in #6448

@bpmct
Copy link
Member

bpmct commented Jun 14, 2023

We haven't found an awesome web-based Windows RDP client that works well inside a workspace (Myrtille works OK, but hasn't been updated in 2 years). For Linux environments, KasmVNC works quite well. We have community example templates for both.

I think a coder desktop <workspace> would be still be significantly helpful over port forwarding and aid the user in the dashboard.

As a workaround, you can add an external coder_app which links to a wiki with some instructions to connect via your desktop RDP:

Screenshot 2023-06-14 at 11 49 12 AM

resource "coder_app" "rdp" {
  agent_id     = coder_agent.main.id
  slug         = "rdp"
  display_name = "RDP instructions"
  external     = true
  url          = "https://coder.com/docs/v2/latest/ides/remote-desktops#rdp-desktop"
  icon         = "https://cdn-icons-png.flaticon.com/512/2933/2933245.png"
}

@spikecurtis spikecurtis added the networking Area: networking label Sep 26, 2023
@bartonip
Copy link
Contributor

Perhaps instead of websockets you could put the VNC stream over WebRTC then everything can just be supported in-browser with no additional client libraries needed?

@sreya
Copy link
Collaborator

sreya commented Apr 4, 2024

Since there's a workaround for this, closing for now

@julianpoy
Copy link

@sreya This page is still linked from https://coder.com/docs/ides/remote-desktops

Perhaps that link should be removed?

@ehfd
Copy link

ehfd commented Jul 9, 2024

Hey, I would like to introduce you to https://github.com/selkies-project/selkies-gstreamer. We are very keen on this to be integrated into Coder with a community template. We recently observed latency levels equal to Moonlight (< 16 ms) when using a hardware encoder, and < 40 ms when using x264. Our CPU consumption over x264 is around 150%, much lower than KasmVNC for 60 fps. We want to know how to get assistance.

selkies-project/selkies#64

@ehfd
Copy link

ehfd commented Jul 9, 2024

CC @bpmct

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api Area: HTTP API enterprise Enterprise-license / premium functionality networking Area: networking site Area: frontend dashboard
Projects
None yet
Development

No branches or pull requests