Skip to content

Adding site map generation #185 #210

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Oct 3, 2014
Merged

Adding site map generation #185 #210

merged 1 commit into from
Oct 3, 2014

Conversation

bjfish
Copy link
Contributor

@bjfish bjfish commented Sep 25, 2014

using the sitemap_generator gem, trigger daily by clock, generated in a sidekiq worker
Some example output

This XML file does not appear to have any style information associated with it. The document tree is shown below.
<urlset xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:image="http://www.google.com/schemas/sitemap-image/1.1" xmlns:video="http://www.google.com/schemas/sitemap-video/1.1" xmlns:geo="http://www.google.com/geo/schemas/sitemap/1.0" xmlns:news="http://www.google.com/schemas/sitemap-news/0.9" xmlns:mobile="http://www.google.com/schemas/sitemap-mobile/1.0" xmlns:pagemap="http://www.google.com/schemas/sitemap-pagemap/1.0" xmlns:xhtml="http://www.w3.org/1999/xhtml" xsi:schemaLocation="http://www.sitemaps.org/schemas/sitemap/0.9 http://www.sitemaps.org/schemas/sitemap/0.9/sitemap.xsd">
<url>
<loc>https://test.com</loc>
<lastmod>2014-09-25T16:53:37+00:00</lastmod>
<changefreq>always</changefreq>
<priority>1.0</priority>
</url>
<url>
<loc>https://test.com/welcome</loc>
<lastmod>2014-09-25T16:53:37+00:00</lastmod>
<changefreq>montlhy</changefreq>
<priority>0.7</priority>
</url>
<url>
<loc>https://test.com/contact_us</loc>
<lastmod>2014-09-25T16:53:37+00:00</lastmod>
<changefreq>montlhy</changefreq>
<priority>0.5</priority>
</url>
<url>
<loc>https://test.com/blog</loc>
<lastmod>2014-09-25T16:53:37+00:00</lastmod>
<changefreq>weekly</changefreq>
<priority>0.7</priority>
</url>
<url>
<loc>https://test.com/api</loc>
<lastmod>2014-09-25T16:53:37+00:00</lastmod>
<changefreq>monthly</changefreq>
<priority>0.5</priority>
</url>
<url>
<loc>https://test.com/faq</loc>
<lastmod>2014-09-25T16:53:37+00:00</lastmod>
<changefreq>monthly</changefreq>
<priority>0.5</priority>
</url>
<url>
<loc>https://test.com/privacy_policy</loc>
<lastmod>2014-09-25T16:53:37+00:00</lastmod>
<changefreq>monthly</changefreq>
<priority>0.2</priority>
</url>
<url>
<loc>https://test.com/tos</loc>
<lastmod>2014-09-25T16:53:37+00:00</lastmod>
<changefreq>monthly</changefreq>
<priority>0.2</priority>
</url>
<url>
<loc>https://test.com/jobs</loc>
<lastmod>2014-09-25T16:53:37+00:00</lastmod>
<changefreq>daily</changefreq>
<priority>0.7</priority>
</url>
<url>
<loc>https://test.com/employers</loc>
<lastmod>2014-09-25T16:53:37+00:00</lastmod>
<changefreq>monthly</changefreq>
<priority>0.7</priority>
</url>
<url>
<loc>https://test.com/p/lwl_wq</loc>
<lastmod>2014-09-25T02:34:32+00:00</lastmod>
<changefreq>weekly</changefreq>
<priority>0.5</priority>
</url>
<url>
<loc>https://test.com/p/yug7ha</loc>
<lastmod>2014-09-25T02:34:33+00:00</lastmod>
<changefreq>weekly</changefreq>
<priority>0.5</priority>
</url>
<url>
<loc>https://test.com/p/-xrl3q</loc>
<lastmod>2014-09-25T02:34:33+00:00</lastmod>
<changefreq>weekly</changefreq>
<priority>0.5</priority>
</url>
<url>
<loc>https://test.com/team/put-a-bird-on-it</loc>
<lastmod>2014-09-25T02:34:34+00:00</lastmod>
<changefreq>weekly</changefreq>
<priority>0.5</priority>
</url>
<url>
<loc>https://test.com/team/team-awesome</loc>
<lastmod>2014-09-25T02:00:30+00:00</lastmod>
<changefreq>weekly</changefreq>
<priority>0.5</priority>
</url>
<url>
<loc>https://test.com/team/team-wonderful</loc>
<lastmod>2014-09-25T02:28:58+00:00</lastmod>
<changefreq>weekly</changefreq>
<priority>0.5</priority>
</url>
<url>
<loc>https://test.com/team/parker-dibbert</loc>
<lastmod>2014-09-25T02:31:33+00:00</lastmod>
<changefreq>weekly</changefreq>
<priority>0.5</priority>
</url>
<url>
<loc>https://test.com/team/glamouros</loc>
<lastmod>2014-09-25T02:52:01+00:00</lastmod>
<changefreq>weekly</changefreq>
<priority>0.5</priority>
</url>
<url>
<loc>
https://test.com/team/glamouros/Manger-of-Cool-wrfhdg
</loc>
<lastmod>2014-09-25T02:52:01+00:00</lastmod>
<changefreq>weekly</changefreq>
<priority>0.5</priority>
</url>
<url>
<loc>https://test.com/bryce</loc>
<lastmod>2014-09-25T02:34:34+00:00</lastmod>
<changefreq>weekly</changefreq>
<priority>0.5</priority>
</url>
<url>
<loc>https://test.com/lisa</loc>
<lastmod>2014-09-25T02:34:34+00:00</lastmod>
<changefreq>weekly</changefreq>
<priority>0.5</priority>
</url>
<url>
<loc>https://test.com/testser</loc>
<lastmod>2014-09-25T04:52:54+00:00</lastmod>
<changefreq>weekly</changefreq>
<priority>0.5</priority>
</url>
<url>
<loc>https://test.com/blog/2012-02-23-hating-on-IE6</loc>
<lastmod>2012-02-23T21:37:05-08:00</lastmod>
<changefreq>weekly</changefreq>
<priority>0.5</priority>
</url>
<url>
<loc>
https://test.com/blog/2012-02-05-the-companies-id-want-to-work-for
</loc>
<lastmod>2012-02-05T20:55:47-08:00</lastmod>
<changefreq>weekly</changefreq>
<priority>0.5</priority>
</url>
<url>
<loc>
https://test.com/blog/2012-01-16-the-hacker-version-of-an-embeddable-social-button
</loc>
<lastmod>2012-01-16T11:02:41-08:00</lastmod>
<changefreq>weekly</changefreq>
<priority>0.5</priority>
</url>
<url>
<loc>
https://test.com/blog/2012-01-10-represent-get-geek-cred-on-your-blog
</loc>
<lastmod>2012-01-10T12:04:36-08:00</lastmod>
<changefreq>weekly</changefreq>
<priority>0.5</priority>
</url>
<url>
<loc>
https://test.com/blog/2012-01-09-starting-off-the-new-year-with-a-bang
</loc>
<lastmod>2012-01-09T14:25:31-08:00</lastmod>
<changefreq>weekly</changefreq>
<priority>0.5</priority>
</url>
<url>
<loc>https://test.com/blog/2011-07-22-gaming-the-game</loc>
<lastmod>2011-07-22T00:09:00-08:00</lastmod>
<changefreq>weekly</changefreq>
<priority>0.5</priority>
</url>
</urlset>

@bjfish bjfish changed the title Adding site map generation Adding site map generation #185 Sep 25, 2014
@bjfish bjfish force-pushed the sitemap_generation branch 2 times, most recently from 1ef3b8f to 4cf8cc0 Compare September 25, 2014 17:09
seuros
seuros reviewed Sep 25, 2014
View reviewed changes
app/workers/sitemap_refresh_worker.rb

def perform
SitemapGenerator::Sitemap.default_host = "https://coderwall.com"
SitemapGenerator::Sitemap.public_path = 'tmp/'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tmp ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@seuros /tmp is recommend here for heroku to temporarily store files before uploading to s3 https://github.com/kjvarga/sitemap_generator/wiki/Generate-Sitemaps-on-read-only-filesystems-like-Heroku

@bjfish
Adding site map generation using the sitemap_generator gem, trigger d…
680c1fd 
…aily by clock, generated in a sidekiq worker
@just3ws
Copy link
Contributor

just3ws commented Oct 3, 2014

Yeah @seuros we need to push sitemaps up to S3 for Heroku. The sitemap generator gem's readme documents the need.

just3ws added a commit that referenced this pull request Oct 3, 2014
@just3ws
Merge pull request #210 from bjfish/sitemap_generation
bf8ab92 
Adding site map generation #185
@just3ws just3ws merged commit bf8ab92 into coderwall:master Oct 3, 2014
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@bjfish @just3ws @seuros