Skip to content

Attempt to correct fullscreen woes #278

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Feb 11, 2022
Merged

Conversation

Xananax
Copy link
Contributor

@Xananax Xananax commented Feb 7, 2022

This PR aims to replace the Godot-side full screen handling with a browser-side fullscreen handling.

This should partially or completely address:

What Was The Challenge?

Two main problems:

  1. There were two competing systems to set the app fullscreen or exit fullscreen. Godot can set the canvas context to fullscreen, the browser can set the window to fullscreen, and both can un-fullscreen the canvas context or the window. Neither system is aware of the other.
  2. When setting the canvas to full screen, Godot's UI interactable area does not correspond to the visual UI on screen. This happens even if the canvas is set as full screen through Javascript. I do not know if the failure is on Godot's end, the browser's end, or due to our CSS (it'd still be a bug, but would explain why it isn't more widespread).

It seems that when setting the document as full screen (as opposed to the canvas context), there's no UI problem.

Note that we cannot call the browser API to set fullscreen from Godot (for security reasons, browsers disable the functionality), so the button to set the document as full screen has to be an html button

This commit addresses problem 1 (#226) by:

  • making sure the browser polls fullscreen changes and dispatches a signal to Godot
  • making sure Godot can listen to the signal

It also addresses #272 by:

  • removing the fullscreen button in the Godot app

Finally, it partially addresses problem 2 (#208) by:

  • removing the Godot fullscreen button in browser contexts
  • removing the fullscreen Godot shortcut in browser contexts
  • adding an overlay html button in the top right corner

"Partially" because there's mention in #208 of UI becoming shifted from clickable areas without any fullscreen happening. I have not been able to reproduce this.

Thoughts For the Future:

  1. We should/could explore the different combinations of Godot stretch modes and CSS, and see if we can fix this in a more idiomatic manner. This will take a long time and require a lot of time-consuming exports to HTML and testing in many browsers. I am not sure it's worth it.
  2. At the moment, Godot-JS translations are distributed across several files. We could reduce code and possibly clarify intent if we moved all JS marshalling in a single autoload (possibly Events.gd?).

This commit aims to replace the Godot-side full screen handling
with a browser-side fullscreen handling.

This should partially or completely address:

- #272
- #226
- #208

## What Was The Challenge?

Two main problems:

1. There were two competing systems to set the app fullscreen or
   exit fullscreen. Godot can set the canvas context to
   fullscreen, the browser can set the window to fullscreen, and
   both can un-fullscreen the canvas context or the window.
   Neither system is aware of the other.
2. When setting the canvas to full screen, Godot's UI
   interactable area does not correspond to the visual UI on
   screen. This happens even if the canvas is set as full screen
   through Javascript. I do not know if the failure is on Godot's
   end, the browser's end, or due to our CSS (it'd still be a
   bug, but would explain why it isn't more widespread).

It seems that when setting the _document_ as full screen (as
opposed to _the canvas context_), there's no UI problem.

Note that we cannot call the browser API to set fullscreen from
Godot (for security reasons, browsers disable the functionality),
so the button to set the document as full screen has to be an
html button

This commit addresses problem 1 (#226) by:
- making sure the browser polls fullscreen changes and dispatches
  a signal to Godot
- making sure Godot can listen to the signal

It also addresses #272 by:
- removing the fullscreen button in the Godot app

Finally, it addresses problem 2 (#208) by:
- removing the Godot fullscreen button in browser contexts
- removing the fullscreen Godot shortcut in browser contexts
- adding an overlay html button in the top right corner
@Xananax
Copy link
Contributor Author

Xananax commented Feb 7, 2022

If we want a more long-term solution, we could consider leaving this PR in the fridge while I explore scaling issues. Alternatively, we could roll it out, then revert it once/if a proper scaling solution is found

@NathanLovato
Copy link
Contributor

If it improves the situation, I'd definitely ship and later consider more issues. We have a limited workforce so I'll take of course full fixes but also partial ones and other improvements.

@NathanLovato
Copy link
Contributor

NathanLovato commented Feb 7, 2022

After testing I found some issues:

  • In brave, the button doesn't appear.
  • In brave and firefox, toggling fullscreen by pressing F11 doesn't work when the Godot app is in focus (after clicking in the app)
  • In firefox, the button appear but as it's in the top-right of the window, it overlaps with the app's interface when the app's full-screen.
    image
  • In firefox, the button works unreliably. The state doesn't always match what you'd expect, like here, it's not full-screen but showing the icon to go out of full-screen.
    image
    Also, in the above screenshot, clicking the button has no effect anymore.

For now, overall, the full-screen toggle part feels broken on my machine. Regarding other issues, I can't say because I can't reproduce them on my two computers, thus can't test them.

The button placement on the page is a bit of an issue: there's nowhere to place it. We can have stuff in every corner of the app. Any idea on how to better integrate it?

@Xananax
Copy link
Contributor Author

Xananax commented Feb 9, 2022

In firefox, the button works unreliably. The state doesn't always match what you'd expect

I see what's happening; Firefox seems to have multiple "types" of fullscreen depending on how it's triggered. I'll work on that.

In brave, the button doesn't appear.

Regarding Brave, are you sure you copied the files to the proper location before? Because it does show on Brave for me:

image

Here's a one-liner to run from project root to compile web:

rm -rf build/web && \
  mkdir -p build/web && \
  godot -v --export "HTML5" build/web/index.html && \
  cp -r html_export/static/* build/web && \
  cd build/web && \
  python -m http.server 8000 && \
  cd -

(This assumes python and godot are available on $PATH)

If the button still doesn't show in Brave (after a cache cleanup, it might be caching the CSS), then can you open your console and see if there's any error message?

In brave and firefox, toggling fullscreen by pressing F11 doesn't work when the Godot app is in focus

Yes, there's nothing to do about this I'm afraid. The Canvas swallows all events. I'm trying to find if I can bubble events from the canvas to the browser, or if I can prevent the canvas from stealing all events.

the button appears in the top-right of the window [and] overlaps with the app's interface

we could move the buttons to the left by the space of one or two button. They'd remain accessible. Alternatively, we can move the fullscreen button to the bottom right

@NathanLovato
Copy link
Contributor

NathanLovato commented Feb 9, 2022

In Brave it might've been a caching issue. I built once and tested both browsers in parallel.

Testing tonight, both in Brave and Firefox, now the button appears. Clicking the fullscreen button after loading the app causes the following error.

In Brave:

Uncaught (in promise) TypeError: Failed to execute 'exitFullscreen' on 'Document': Document not active
    at HTMLButtonElement.toggle (bootstrap.js:408:20)

In Firefox:

Uncaught (in promise) TypeError: Not in fullscreen mode
    toggle http://0.0.0.0:8000/bootstrap.js:408

And fullscreen doesn't toggle, unfortunately.

When first making the browser full-screen and then clicking the button to get out of fullscreen, the same error appears.

@Xananax
Copy link
Contributor Author

Xananax commented Feb 9, 2022

After more investigation; I have no documentation about this, it's entirely conjecture, but it seems that all browsers have 2 "fullscreen" mode, which are not doing the same thing at all.

TL,DR: we can't do much about it. Jump to the bottom for mitigations.

longer explanation:

I'll call them mode API and mode native. The below explains the differences

Mode API

The fullscreen triggered by the Javascript API takes an element (normally, a video, such as a Youtube video) and fills the screen with it.

Mode Native

The fullscreen mode triggered by F11 maximizes the browser chrome, removes bars (but leaves the sidebar if it is open). This mode is clearly different from mode 1.

Some key differences:

Mode API:

  1. a notification appears on top saying "so and so is now full screen"
  2. ESC automatically escapes full screen

Mode Native:

  1. an animation plays, where the bars slowly shrink out of view
  2. the top address bar shows when the mouse hits to top part of the screen
  3. ESC does not exit full screen, pressing F11 again is mandatory

Some similarities

both modes:

  1. set the css mode to fullscreen, which means that my media query @media all and (display-mode: fullscreen) { unfortunately runs on both, and I can't differentiate them
  2. Confusingly, you can do both F11 and then press a maximize button: try opening a youtube page, then press F11 , then press the maximize button in the video. If you then press ESC, you'll go back to native fullscreen mode.
  3. F11 exits both fullscreen modes.

What doesn't work:

  1. hijacking f11 to trigger our own fullscreen; browser security prevents it, it has to be a user pressing a button
  2. doing document.body.requestFullscreen or any other combination; they all trigger mode API; window.requestFullscreen doesn't exist. There doesn't seem a way to trigger mode Native.
  3. as you noticed, if I try to exit full screen from mode Native, the browser throws.

Prior literature

This fullscreen issue isn't discussed anywhere, but I found references to the difficulty of fullscreen in general;

Checking the screenful demo or any youtube video will demonstrate the two different behaviors.

Pictures attached below, open to see

Screenfull Normal, Mode API, Mode Native

screenfull normal

screenfull mode api

screenfull mode native

Youtube Normal, Mode API, Mode Native

youtube normal

youtube mode api

youtube mode native

Mitigations:

  1. remove our fullscreen button entirely, let the browser handle everything
  2. disable F11. While we cannot trigger fullscreen from a shortcut, we can simply prevent it from working
  3. do it half-half, and hide the button if the fullscreen was not triggered by a press. This is detectable fairly reliably

@NathanLovato
Copy link
Contributor

What would you recommend? We need to find a way to solve the issue with the UI not matching where the user has to click, and ideally make full-screen toggles "just work" as far as the user is concerned. Whatever gets closest to fixing that seems good to me.

@Xananax
Copy link
Contributor Author

Xananax commented Feb 9, 2022

I'll try doing 2 & 3.

  1. if user clicks, the user can click again. the user can also press escape to exit
  2. if the user presses F11, then nothing happens
  3. if the user opens the menu, and presses full screen from there, then the button hides. The user can then access the menu again to exit

Potential issues:

  1. top buttons need to be moved to the left
  2. someone could have binded fullscreen to a different key
  3. someone could click the button, then press in the black bars, then trigger native fullscreen, which means they'd have to exit two full screen context. It's a stretch, but technically possible.

Other possibility that I didn't mention:

We could also investigate more fullscreen from Godot and why it isn't working. I did try a bit, and the same problems occurs with our stretchy CSS removed, so it seems our CSS is unrelated to the problem (but I could investigate more)

@NathanLovato
Copy link
Contributor

We could probably get in touch with Faless for extra insights and for insights about #253... I don't know if he'd have time but we could also hire him for that, as he's the person who should know the most about Godot's HTML5 exports. He could probably solve the issue much for efficiently than us.

@Xananax
Copy link
Contributor Author

Xananax commented Feb 9, 2022

If 2 & 3 don't work, then the optimal way to deal with this is probably to simply remove the option of full screen completely. Someone wants to do that, they can use the browser functionality.

We could research this more, but the energy cost is starting to be disproportionate in regards to the yield

@NathanLovato
Copy link
Contributor

Definitely yes. I'm personally fine with the state of full-screen, but it's an issue if the interaction zones don't update for some users.

Note that some users reported this happened to them on displays with ratios that are not 16:9. Widescreens, ultrawide displays. If your display like this too?

this commit also introduces a neat debugging tool for the browser

Append `?debug` to the url to get debug strings in the console.
Use `?debug=fullscreen` to only get fullscreen debug.
Output multiple modules with `?debug=fullscreen,loader`
This way, the user can press enter or space
@Xananax
Copy link
Contributor Author

Xananax commented Feb 10, 2022

Current behavior is:

(note: F11 is disabled, but I'll still use F11 to refer to the shortcut, in case someone changed the default binding)

  • pressing F11 hides until getting out of fullscreen. It forces the user to press F11 again.
  • pressing , then F11, also hides. Pressing F11 again exits both fullscreen modes.

In other words, there's no way anymore for a user to layer the two modes anymore (hopefully).

Additionally:

  • As said earlier, pressing F11 doesn't trigger fullscreen anyway
  • Instead, pressing F11 focuses . This way the user can press or to toggle it.
  • the Godot fullscreen button now still exists, but doesn't do anything and is invisible. This leaves space in the top right (I now reflect that it'd still be reachable by tabbing to it...?)

Side improvements:

this commit also introduces a neat debugging tool for the browser

Append ?debug to the url to get debug strings in the console (e.g.: gdquest.github.io/learn-gdscript/?debug#rest/of/url)
You can filter the modules; use ?debug=fullscreen to only get fullscreen debug. Output multiple modules with ?debug=fullscreen,loader

General status

It's resolved enough imho, with the caveat that I cannot reproduce the misalignement outside of fullscreen. My screen isn't 16:9, but I've tried simulating it through different combinations of browser window size and zoom, and I cannot repro (on Firefox, Chromium, Brave).

@NathanLovato
Copy link
Contributor

It's working well now.

I found one edge case.

In brave, if you press F11 the button gets in focus but doesn't take you to fullscreen. If you then click it once while in focus, you get to fullscreen. Click the button again and you get out of fullscreen. After that, clicking the button or pressing F11 doesn't do anything anymore

This issue only occurs when first pressing F11 to make the button in focus.

In Firefox, pressing F11 focuses the button but doesn't toggle fullscreen. However, the issue you see on Brave doesn't occur: toggling fullscreen by clicking always works. I'll also reinstall chrome to start testing there systematically, as it's probably the most used browser by our students.

@NathanLovato
Copy link
Contributor

Same result as Brave when testing in Google chrome

@Xananax
Copy link
Contributor Author

Xananax commented Feb 11, 2022

I was able to repro once, but no more. It's very strange. The strangest part is I got this on recording because I wanted to show you I couldn't repro. So I have the exact steps, but despite trying them again, as well as a lot of variations, I couldn't reproduce it another time.

I tried nonetheless to guess what happened and I may have fixed it. If your can consistently repro, may I ask you to run the app with this url: http://localhost:8000/?debug=fullscreen

Do not open the inspector (in case that plays a role in any sort of way), then, if you can reproduce the bug, then open the inspector and send me what's written there.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants