|
1 | 1 | +++
|
2 |
| -title = "Extending the Coreutils project - Rewriting base tools in Rust " |
| 2 | +title = "Extending the Coreutils project - Rewriting base tools in Rust" |
3 | 3 | draft = true
|
4 | 4 | date = 2024-10-01
|
5 | 5 | authors = ["Sylvestre Ledru", "Terts Diepraam"]
|
6 | 6 | +++
|
7 | 7 |
|
8 |
| -Over the last 4 years, we have been working at reimplementing some of the key Linux tools in Rust. We started with the [Coreutils](https://github.com/uutils/coreutils) and [findutils](https://github.com/uutils/findutils). |
| 8 | +Over the last four years, we have been working on reimplementing some of the key Linux tools in Rust. We started with the [Coreutils](https://github.com/uutils/coreutils) and [findutils](https://github.com/uutils/findutils). |
9 | 9 |
|
10 | 10 | As we approach feature parity with the GNU Coreutils implementation, and as its adoption in production environments continues to expand, we have been thinking about what is next.
|
11 | 11 |
|
12 | 12 | Given the overwhelming positive feedback around this initiative, we are going to extend our efforts to rewrite other parts of the modern Linux/Unix/Mac stack in Rust (still with Windows support in mind when relevant).
|
13 |
| -We also noticed a lot of contributions on these projects coming from a lot of different hackers (475 different contributors on Coreutils!). |
14 |
| -With the growing enthusiasm for Rust and the eagerness to learn it, now is the best time to push this project. We think that rewriting in Rust will help with the longer term maintenance of the ecosystem, ensuring it stays robust, safe and welcoming for new generations of contributors. |
| 13 | +We also noticed a lot of contributions on these projects coming from a diverse group of hackers (475 different contributors on Coreutils alone!). With the growing enthusiasm for Rust and the eagerness to learn it, now is the best time to push this project forward. Rewriting in Rust will help with the long-term maintenance of the ecosystem, ensuring it stays robust, safe, and welcoming for new generations of contributors. |
15 | 14 |
|
16 |
| -For the next projects, we are using the same approach: dropped-in replacement of the GNU C implementation. |
17 |
| -For now, we are going to focus on: |
18 |
| -* [diffutils](https://github.com/uutils/diffutils) (transferred by Michael Howell) - [Good first issues](https://github.com/uutils/diffutils/labels/good%20first%20issue) |
19 |
| - Almost ready |
20 |
| -* [procps](https://github.com/uutils/procps) - [Good first issues](https://github.com/uutils/procps/labels/good%20first%20issue) |
21 |
| - 5 programs started |
22 |
| -* [acl](https://github.com/uutils/acl) - [Good first issues](https://github.com/uutils/acl/labels/good%20first%20issue) |
23 |
| - The 3 started |
24 |
| -* [util-linux](https://github.com/uutils/util-linux) - [Good first issues](https://github.com/uutils/util-linux/labels/good%20first%20issue) |
25 |
| - A couple programs started |
26 |
| -* [bsdutils](https://github.com/uutils/bsdutils) - [Good first issues](https://github.com/uutils/bsdutils/labels/good%20first%20issue) |
27 |
| - One program started |
28 |
| -* [login](https://github.com/uutils/login/) - [Good first issues](https://github.com/uutils/login/labels/good%20first%20issue) |
29 |
| - Just the skeleton |
30 |
| -* [hostname](https://github.com/uutils/hostname/) - [Good first issues](https://github.com/uutils/hostname/labels/good%20first%20issue) |
31 |
| - Almost empty |
| 15 | +### Next Steps |
32 | 16 |
|
| 17 | +For the next phase of the project, we are adopting the same approach: a drop-in replacement of the GNU C implementations. Here's what's coming next: |
33 | 18 |
|
34 |
| -These packages are part of the essential list on Debian & Ubuntu. |
| 19 | +- **[diffutils](https://github.com/uutils/diffutils)** (transferred by Michael Howell) - Almost ready. |
| 20 | +- **[procps](https://github.com/uutils/procps)** - Five programs started. |
| 21 | +- **[acl](https://github.com/uutils/acl)** - Three programs started. |
| 22 | +- **[util-linux](https://github.com/uutils/util-linux)** - A few programs started. |
| 23 | +- **[bsdutils](https://github.com/uutils/bsdutils)** - One program started. |
| 24 | +- **[login](https://github.com/uutils/login/)** - Skeleton started. |
| 25 | +- **[hostname](https://github.com/uutils/hostname/)** - Almost empty. |
35 | 26 |
|
| 27 | +These packages are part of the essential list for Debian and Ubuntu, and we're excited to push their Rust reimplementation further. |
36 | 28 |
|
37 |
| -Sylvestre Ledru, Terts Diepraam and Daniel Hofstetter |
| 29 | +### GSoC 2024 Participation |
38 | 30 |
|
39 |
| -FAQ |
40 |
| -### |
| 31 | +This year, we had the privilege of mentoring three students during Google Summer of Code (GSoC) 2024, who contributed to the project in remarkable ways: |
41 | 32 |
|
42 |
| -How long is it going to take? |
43 |
| ------------------------------ |
| 33 | +1. **Sreehari Prasad TM** worked on improving the support of Rust-based coreutils in Debian. His focus was on making uutils compatible with the GNU coreutils test suite. Sreehari resolved most of the failing tests for the `cp`, `mv`, and `ls` utilities and significantly enhanced compatibility with Debian. |
44 | 34 |
|
45 |
| -Some programs like diff, acl or hostname should be completed quickly. |
46 |
| -Others will take years to be completed. |
| 35 | +2. **Hanbings** tackled the implementation of key GNU `findutils` utilities like `xargs`, `find`, `locate`, and `updatedb` in Rust. His work focused on improving compatibility with the GNU suite while enhancing performance, resulting in major progress on missing features and test code. |
47 | 36 |
|
48 |
| -Do you hate the GNU project? |
49 |
| ----------------------------- |
| 37 | +3. **[Third GSoC Student]** worked on implementing the `procps` suite, which includes utilities like `slabtop`, `top`, `pgrep`, `pidof`, `ps`, `pidwait`, and `snice`. This project involved cross-platform support and optimization of performance, focusing first on Linux implementations with plans to extend to other Unix systems in the future. |
50 | 38 |
|
51 |
| -Not at all: we often contribute to the upstream implementations when we identify issues, add missing tests, etc. |
| 39 | +These students made significant contributions to the Rust coreutils project, pushing forward both feature parity and overall performance improvements. |
52 | 40 |
|
| 41 | +### FAQ |
53 | 42 |
|
54 |
| -Why the MIT License? |
55 |
| --------------------- |
| 43 | +**How long is it going to take?** |
56 | 44 |
|
57 |
| -For consistency purposes and not interested by a license debate, we are going to use the MIT license just like the Coreutils. |
| 45 | +Some programs, like `diff`, `acl`, or `hostname`, should be completed quickly, while others will take years to finish. |
58 | 46 |
|
| 47 | +**Do you hate the GNU project?** |
59 | 48 |
|
60 |
| -The binaries are too big? |
61 |
| -------------------------- |
| 49 | +Not at all. We often contribute to the upstream implementations when we identify issues, add missing tests, and so on. |
62 | 50 |
|
63 |
| -Does it really matter? Yes, Rust binaries are usually bigger than GNU but we don't think it is a blocker, even for embedded devices. |
| 51 | +**Why the MIT License?** |
64 | 52 |
|
| 53 | +For consistency purposes. We're not interested in a license debate and will continue to use the MIT license, as we did with Coreutils. |
65 | 54 |
|
66 |
| -Is it really safer? |
67 |
| -------------------- |
| 55 | +**The binaries are too big. Does it really matter?** |
68 | 56 |
|
69 |
| -While Rust is clearly a better language than C/C++ for security, these programs are often very safe. It isn't a key argument for us for |
70 |
| -this rewrite. |
| 57 | +Yes, Rust binaries are generally bigger than GNU's, but we don't think it's a blocker, even for embedded devices. |
71 | 58 |
|
72 |
| -What about performances? |
73 |
| ------------------------- |
| 59 | +**Is it really safer?** |
74 | 60 |
|
75 |
| -Although we haven't entered the optimization phase, Rust's modern design inherently facilitates some straightforward performance improvements. |
| 61 | +While Rust is better than C/C++ for security, these programs are often already very safe. Security is not the key argument for us in this rewrite. |
| 62 | + |
| 63 | +**What about performance?** |
| 64 | + |
| 65 | +Although we haven't started optimizing yet, Rust's design facilitates performance improvements naturally. We are confident that, in time, these tools will match or exceed the performance of their GNU counterparts. |
0 commit comments