Skip to content

Expose display’s root_group, add function to resize REPL terminal #5954

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Feb 2, 2022

Conversation

kmatch98
Copy link

This exposes the REPL's display group elements as supervisor.splash. Also, this adds a function to resize the dimensions of the REPL terminal window. This allows the user to view the REPL and console print() output as well as view any display elements at the same time.

Warning: This is somewhat dangerous since exposing the REPL display elements allows the the REPL to be relocated where it is no longer visible.

Here is an example creating a split screen with the REPL at the bottom of the display and displayio elements at the top of the screen.

IMG_2486.MOV
import board
import displayio
import supervisor
import time


display=board.DISPLAY

# Create a bitmap with two colors
bitmap1 = displayio.Bitmap(display.width//4, display.height//4 - 10, 2)

# Create a two color palette
palette1 = displayio.Palette(2)
palette1[0] = 0xff00ff
palette1[1] = 0xffffff

# Create a TileGrid using the Bitmap and Palette
tile_grid1 = displayio.TileGrid(bitmap1, pixel_shader=palette1)


# Create a bitmap with two colors
bitmap2 = displayio.Bitmap(display.width//4, display.height//4 - 10, 2)

# Create a two color palette
palette2 = displayio.Palette(2)
palette2[0] = 0x00ffff
palette2[1] = 0xffffff

# Create a TileGrid using the Bitmap and Palette
tile_grid2 = displayio.TileGrid(bitmap2, pixel_shader=palette2)
tile_grid2.x=display.width//4

# Create a bitmap with two colors
bitmap3 = displayio.Bitmap(display.width//4, display.height//4 - 10, 2)

# Create a two color palette
palette3 = displayio.Palette(2)
palette3[0] = 0xffff00
palette3[1] = 0xffffff

# Create a TileGrid using the Bitmap and Palette
tile_grid3 = displayio.TileGrid(bitmap3, pixel_shader=palette3)
tile_grid3.x=2*display.width//4

# Create a Group
mygroup = displayio.Group()

print()
print()
print()
print("REPL control test: Taking control of REPL group")
time.sleep(3)

# Note: You must "display.show" your own group before adding the supervisor.splash to your own group.
# Reason: When displaying the normal REPL (for example with display.show(None), the supervisor.splash
# group is already in a group that is displayed.  To remove the supervisor.splash from the displayed group,
# you first have to display.show some other group, doing that will remove the supervisor.splash from its group
# and allow you to append it to your own group.

display.show(mygroup)
# add the supervisor.splash group to the displayed group.
mygroup.append(supervisor.splash)

# resize the supervisor.splash group pixel dimensions, make it half the display height.
supervisor.reset_terminal(display.width, display.height//2)

# relocate the supervisor.splash group on the display, moving it half-way down the display
supervisor.splash.y=display.height//2
print("Resize and move the splash screen")
time.sleep(2)

# demonstrate how print statements scroll on the REPL/console display
print("Add some prints to show terminal scrolling:")
time.sleep(0.3)
for i in range(6):
	print("Line: {}".format(i))
	time.sleep(0.3)
time.sleep(1.5)

# Add bitmap rectangles to the display group
print("Add some displayio bitmap rectangles:")
time.sleep(1)

# Add the TileGrid to the Group
print("Add a pink rect.")
mygroup.append(tile_grid1)
time.sleep(1)
print("Add a cyan rect.")
mygroup.append(tile_grid2)
time.sleep(1)
print("Add a yellow rect.")
mygroup.append(tile_grid3)
time.sleep(1)

# print some more lines to show the console text scrolling, with 
# the bitmaps still up on the upper half of the display
for i in range(6):
	print("Another line feed: {}!".format(i))
	time.sleep(0.3)
time.sleep(3)
print("Resetting the REPL and ending the code")
time.sleep(1)

# Before finishing this code, set the terminal size to the full display pixel dimensions
supervisor.reset_terminal(display.width, display.height)

Copy link
Member

@tannewt tannewt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like this functionality but I've got some questions:

  • Could we add the ability to get the current Group from Display instead of having a separate accessor?
  • Could we reset the location of the terminal at a certain point in case things go awry?
  • Would it be more versatile to provide the serial input to terminal into userland so that it can create it's own Terminal?

@anecdata
Copy link
Member

anecdata commented Jan 31, 2022

This is a cool feature, and probably faster than approximating it with a partial-screen terminalio.Terminal(which also probably needs some gyrations to get print() statements to be natural).

@kmatch98
Copy link
Author

Thanks for the constructive comments.

  • being able to access the current Group would be more general. Yet if you’re somewhere already deep in your display code, it would require you to do a display.show(None) to access a handle to the splash group. Probably not a huge deal and you could turn off auto_refresh so it would be hidden to the user. Let me know your decision and I’ll make edits.
  • I assume that when code.py stops that it essentially does a display.show(None). I’ll hunt for where the display.show(None) happens and that can reset the display to its original position and size.
  • I’m with @anecdata on this one. A user can build their own terminal, but if the REPL is already there, that makes it a lot simpler to use.
  • One other idea I toyed with is whether resizing the REPL should also forward over any text data from the previous incarnation of the Terminal tilegrid. Currently the tilegrid is obliterated and a fresh one is created. However that would require allocating the new tilegrid memory, copying the tilegrid data that will fit and then deallocating the previous tilegrid. Is it worth this effort/code space to keep the REPL text even when the size changes?

@kmatch98 kmatch98 force-pushed the repl_wrangler branch 4 times, most recently from 5660599 to 5098537 Compare February 1, 2022 02:55
@kmatch98
Copy link
Author

kmatch98 commented Feb 1, 2022

I totally messed up my commits. I’ll see if I can enlist help on discord to get the revision sorted out.

@kmatch98
Copy link
Author

kmatch98 commented Feb 1, 2022

Ok, this is ready for another review:

  • Created an accessor for a displays root_group.
  • Resets the REPL size and position when display.show(None) is called.

Some of the builds are failing. I'm unsure how to resolve this.

@FoamyGuy
Copy link
Collaborator

FoamyGuy commented Feb 1, 2022

This is awesome! I have not tried it out on a device yet, but I'm looking forward to it.

Looking over the code and example presented above my only question ATM is around naming.

Is the "splash" term previously used somewhere for this concept (or something similar)?

I have seen it used as the name for a Group object in some example code scripts. But I've always thought that when originally written the object may have been named that way because it was used for a "splash screen" which mobile apps and other games often have. (Shows the Logo or some other artwork for a few seconds while more things are being loaded).

Maybe there is some context for the term that I am missing though. I have seen it used in cases even when the object doesn't represent a "splash screen" type element.

If "Splash" has a specific meaning that relates to this concept then I think it's okay to use it. But if it was chosen here because there are many examples that use it for their Group variable, I might lean toward trying to come up with something more descriptive or specific to our usage here.

@kmatch98
Copy link
Author

kmatch98 commented Feb 1, 2022

Thanks @FoamyGuy for the good feedback. Per @tannewt comments above I modified the functionality and naming. To clarify, here is the current status of what is added and the names.

  • display.root_group: the root displayio Group that is currently displayed. To access the “splash” screen group (with Blinka and the REPL terminal output) you first have to do display.show(None) and then procure the display.root_group. If you don’t want that to disturb the display you can turn auto_refresh to false.
  • supervisor.reset_terminal(x_pixels, y_pixels): function to resize the REPL terminal.
  • finally, I added some code to reset the REPL to its default position and size whenever display.show(None) is called. This ensures that the REPL returns to its expected form whenever code.py ends.

@kmatch98 kmatch98 changed the title Expose splash group, add function to resize REPL terminal Expose display’s root_group, add function to resize REPL terminal Feb 1, 2022
@kmatch98 kmatch98 changed the title Expose display’s root_group, add function to resize REPL terminal Expose display’s root_group, add function to resize REPL terminal Feb 1, 2022
@tannewt
Copy link
Member

tannewt commented Feb 1, 2022

  • One other idea I toyed with is whether resizing the REPL should also forward over any text data from the previous incarnation of the Terminal tilegrid. Currently the tilegrid is obliterated and a fresh one is created. However that would require allocating the new tilegrid memory, copying the tilegrid data that will fit and then deallocating the previous tilegrid. Is it worth this effort/code space to keep the REPL text even when the size changes?

I think this would be way more work than it's worth. The moment you change the width, you'd then need to rewrap all of the text. Nevermind the VT100 commands that can actually navigate the grid and modify it. Obliterate is my preference.

Copy link
Member

@tannewt tannewt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the iteration. It looks good!

The builds are failing because supervisor_stop_terminal is only linked in for boards that support displayio. I've added a suggestion that should fix it.

I agree the term "splash" isn't ideal but that's what I used when I added it. Maybe "serial_terminal" is clearer? I don't care if you change it or not in this PR. The term is only used internally so we're not committing to it.

kmatch98 and others added 2 commits February 1, 2022 14:36
Co-authored-by: Scott Shawcroft <scott@tannewt.org>
Co-authored-by: Scott Shawcroft <scott@tannewt.org>
@kmatch98 kmatch98 requested a review from tannewt February 1, 2022 23:34
Copy link
Collaborator

@FoamyGuy FoamyGuy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I attempted to build from this branch and run on a PyPortal but it seems that I'm getting a hard crash.

Immediately after the UF2 finishes pasting, the device reboots as normal. And for just one second I can see the serial output on the display as normal:

pyportal_troubleshoot

But immediately after this the display goes dark and I never see CIRCUITPY or the serial port connect from the PC side. It only shows this very briefly after pasting UF2 and only once. After it disappears the first time then it doesn't ever get shown again until changing back to 7.1.1 and then back to the version from this branch.

On subsequent resets from the button and unplug/replug nothing happens on the display it stays dark. But the onboard LEDs do seem blink as normal.

@kmatch98
Copy link
Author

kmatch98 commented Feb 2, 2022

@FoamyGuy this pyportal issue was resolved by
#5951

I pulled the repo before that PR was merged so it’s not incorporated in this repo. Do I need to merge in those changes?

@FoamyGuy
Copy link
Collaborator

FoamyGuy commented Feb 2, 2022

@kmatch98 ah I see! I don't think you need to merge it to here(I believe github works out the differences correctly in most cases), but I am not certain. I'll try it again tomorrow and merge that PR into my local repo to build.

@kmatch98
Copy link
Author

kmatch98 commented Feb 2, 2022

Fixed an erroneous documentation string. Local docs build ok now. This is ready for another review.

This failed for one board, uses 52 bytes too many. Not sure how to resolve this one. Suggestions welcome.

@tannewt
Copy link
Member

tannewt commented Feb 2, 2022

@dhalbert did you reduce the datum_imu board size already?

Copy link
Member

@tannewt tannewt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good to me! I think the one board that's too big may be fixed by another PR so we can merge this.

@dhalbert
Copy link
Collaborator

dhalbert commented Feb 2, 2022

@dhalbert did you reduce the datum_imu board size already?

Yes, fixed by 0d43e3e in #5969.

@dhalbert dhalbert merged commit 4dc9b00 into adafruit:main Feb 2, 2022
@kmatch98
Copy link
Author

kmatch98 commented Feb 2, 2022

Here's the latest demo code for moving the REPL position and adding some displayio objects:

import board
import displayio
import supervisor
import time


display=board.DISPLAY

# Create a bitmap with two colors
bitmap1 = displayio.Bitmap(display.width//4, display.height//4 - 10, 2)

# Create a two color palette
palette1 = displayio.Palette(2)
palette1[0] = 0xff00ff
palette1[1] = 0xffffff

# Create a TileGrid using the Bitmap and Palette
tile_grid1 = displayio.TileGrid(bitmap1, pixel_shader=palette1)


# Create a bitmap with two colors
bitmap2 = displayio.Bitmap(display.width//4, display.height//4 - 10, 2)

# Create a two color palette
palette2 = displayio.Palette(2)
palette2[0] = 0x00ffff
palette2[1] = 0xffffff

# Create a TileGrid using the Bitmap and Palette
tile_grid2 = displayio.TileGrid(bitmap2, pixel_shader=palette2)
tile_grid2.x=display.width//4

# Create a bitmap with two colors
bitmap3 = displayio.Bitmap(display.width//4, display.height//4 - 10, 2)

# Create a two color palette
palette3 = displayio.Palette(2)
palette3[0] = 0xffff00
palette3[1] = 0xffffff

# Create a TileGrid using the Bitmap and Palette
tile_grid3 = displayio.TileGrid(bitmap3, pixel_shader=palette3)
tile_grid3.x=2*display.width//4

# Create a Group
mygroup = displayio.Group()

print()
print()
print()
print("REPL control test: Taking control of REPL group")
time.sleep(3)


# clear the display to the REPL 
display.show(None)
splash = board.DISPLAY.root_group # this gets the current root_group, the REPL

# Note: You must "display.show" your own group before adding the splash to your own group.
# Reason: When displaying the normal REPL (for example with display.show(None), the splash
# group is already in a group that is displayed.  To remove the splash from the displayed group,
# you first have to display.show some other group, doing that will remove the splash from its group
# and allow you to append it to your own group.	
display.show(mygroup)

# resize the supervisor.splash group pixel dimensions, make it half the display height.
supervisor.reset_terminal(display.width, display.height//2)

# relocate the supervisor.splash group on the display, moving it half-way down the display
splash.y=display.height//2
print("Resize and move the splash screen")

# append the supervisor.splash group to the displayed group.
mygroup.append(splash)
time.sleep(2)

# demonstrate how print statements scroll on the REPL/console display
print("Add some prints to show terminal scrolling:")
time.sleep(0.3)
for i in range(6):
	print("Line: {}".format(i))
	time.sleep(0.3)
time.sleep(1.5)

# Add bitmap rectangles to the display group
print("Add some displayio bitmap rectangles:")
time.sleep(1)

# Add the TileGrid to the Group
print("Add a pink rect.")
mygroup.append(tile_grid1)
time.sleep(1)
print("Add a cyan rect.")
mygroup.append(tile_grid2)
time.sleep(1)
print("Add a yellow rect.")
mygroup.append(tile_grid3)
time.sleep(1)

# print some more lines to show the console text scrolling, with 
# the bitmaps shown on the upper half of the display
for i in range(6):
	print("Another line feed: {}!".format(i))
	time.sleep(0.3)
time.sleep(3)
print("Ending the code")
time.sleep(1)

@matemaciek
Copy link

matemaciek commented Mar 27, 2024

splash.y=display.height//2

On CP 9.0.1 this line results in RuntimeError: Read-only. Is there any other way of shifting the terminal on display now?

@jepler
Copy link

jepler commented Mar 27, 2024

@matemaciek Please file a new PR about this message. This most likely is an undesired side-effect of #8923 that should be fixed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants