Skip to content

LLVM Discourse forums are not indexed by Google, is our robots.txt configured correctly? #156790

@rnk

Description

@rnk

Hey folks, when I search for highly unique LLVM Discourse posts by name, such as this one, using the text "ARM32 Compiler Support for Skip Fault Mitigation", I get no results from discourse.llvm.org when I use Google, but I get the result I'm looking for from DuckDuckGo and Bing.

I think our users and developers in affiliated projects (Rust, Swift, whoever) would greatly benefit from being able to search up information from LLVM RFC threads to better understand why certain design choices were made, so it would be good if we could fix this.

I looked at our robots.txt, and I see a bunch of special Googlebot rules. They look correct to me, but I'm not an expert.

Since Google happens to be my employer, I may circulate this issue internally and get better results that way, but I wanted to start by publicly documenting the issue.

Metadata

Metadata

Assignees

No one assigned

    Labels

    discourseBugs or requests for LLVM DiscourseinfrastructureBugs about LLVM infrastructurequestionA question, not bug report. Check out https://llvm.org/docs/GettingInvolved.html instead!

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions