Some comments from my side:

Packages sizes for v21.1.1 are

So ptb is <1.5 times as large as ptb-raw while still behing well below 0.5MB in source / 1MB in whl. IMO footprint size concernes should not be too big.

I'm in favor of deprecating ptb-raw. Even with installing ptb, you can still use the telegram module independently from the telegram.ext module. Aditionally, this removes some maintenance overhead.
Since our stability policy doesn't strictly say anything about deprecating the whole package and b/c there is a drop-in replacement available via ptb, I don't think we need to stick to the two-major-versions cycle. Instead my first idea would be

• We announce deprecation of ptb-raw in the channel
• with the next possible release, we issue a warning in telegram/__init__.py in case telegram.ext is not found. This warning points users to the channel post and tells them to use ptb instead.
• the following release does no longer update ptb-raw

With abandoning ptb-raw, we can then easily move to a pyproject.toml build setup, which is more modern and may offer some additional flexibility.
I recommend to use hatch(ling) as build requirement. It's a 1st party library, fully PEP-compliant and I've already a tiny bit of experience with it.
As a first step we can simply manually fill pyproject.toml file and then decide later if we want to use (parts of) hatch for further packaging steps (e.g. updating the version numbers)

related: #4282