Skip to content

Feature Request: More options for controlling automatic shutdown #5600

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
NiklasRosenstein opened this issue Jan 6, 2023 · 4 comments
Closed
Assignees
Labels
stale This issue is like stale bread.

Comments

@NiklasRosenstein
Copy link
Contributor

The current situation for auto shutdown appears to be:

When you create a Coder Workspace, it will be scheduled to automatically shut down after the TTL set in the template has passed, or what the user has configured in the Workspace schedule. If Coder detects any activity (through SSH/Port forwarded connections?), and the Workspace would be shut down in under one hour, it will extend the shut down time to one hour.

Sometimes the "Stops at" information is not shown on a Workspace, and I'm not sure why, but when it is, you can only use it to reduce the time at which the Workspace is stopped, or increase it back up to the max TTL.

What I would really like to see is:

  • Ability to increase the lifetime above the max TTL. This could be achieved by introducing a default TTL that is used when the Workspace is created, and users can increase it up to the max TTL.
  • Ability to set a shutdown by date & time instead of by "+ hours".
  • Allow setting a fixed time in the schedule at which the workspace will be shutdown (unless activity is detected or the lifetime was manually extended), similar to how we can currently specify startup times.

This would be great for AI research where the allocated resources are very expensive. Having a low "default TTL" ensures that we shut down the workspace sooner than later. Currently I wouldn't be able to set the max TTL below 48 hours to allow me to allow the workspace can actually live for this long (to complete scheduled trainings that might take >24h to complete), but this also means that unless I remember to shut down the workspace when I don't need it to run over night, it will just keep running.

Related side question: Is it possible to stop the Workspace from inside the workspace without requiring the user to run coder login from inside the Workspace first? This could enable to set up a Cron-job that would periodically check resource usage and stop the workspace based on that information.

@matifali
Copy link
Member

matifali commented Jan 6, 2023

Doing stop from inside the workspace for example when training ends seems like great idea. This will save a lot of money and resources.
May be a python package that can trigger workspace stop when called at the end of a script.

@mattlqx
Copy link

mattlqx commented Jan 7, 2023

I think if losing contact with an agent would trigger a Terraform workspace refresh (or doing it on some sort of routine basis), it would suit these situations well where the compute resource changes state outside of an action in Coder. I have a use-case for this as well where I want aggressive background processes controlling high-cost instances.

@ammario ammario self-assigned this Jan 16, 2023
@ammario ammario removed their assignment Feb 6, 2023
@github-actions
Copy link

This issue is becoming stale. In order to keep the tracker readable and actionable, I'm going close to this issue in 7 days if there isn't more activity.

@github-actions github-actions bot added the stale This issue is like stale bread. label May 15, 2023
@matifali
Copy link
Member

@mattlqx, I believe this can be achieved by using the workspaces API

Closing as completed
cc: @bpmct , @johnstcn

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
stale This issue is like stale bread.
Projects
None yet
Development

No branches or pull requests

5 participants