Excluding member-specific pages from sitemap Answered

Any ideas on how to exclude member-specific pages from sitemap? Our customers have custom dashboards and we use member-specific page generated by Memberstack IDs. It seems like they can be easily breached by malware. I tried using robots.txt but it didn’t work.

Comments

5 comments

  • Comment author
    Chukwudi Onyekwere

    Hi Alican,

    You can choose to create your own Sitemap on Webflow as shown in their article here (https://university.webflow.com/lesson/sitemap) or if you aren't using Webflow, some websites like https://www.xml-sitemaps.com/ could be used to generate a Sitemap for your website, then modify to your need.

    Alternately just exclude that page from search indexing by using the <meta name="robots" content="noindex" /> tag in the head of that document.

    I hope this helps.

    0
  • Comment author
    Alican Avcuoğlu

    The challenge in creating my own Sitemap is that I have many users who are creating their own pages during sign-up, which means the Sitemap needs to be updated automatically. I used the “noindex” meta tag, which may prevent the pages from being indexed by search engines, but it does not affect the Sitemap.

    0
  • Comment author
    Chukwudi Onyekwere

    Well, probably writing a cron job which runs on daily basis on your server that would generate a new sitemap according to all newly generated content, return the sitemap in the form of an xml file, and would then overwrite the previous one could work in this case. I don't think changing sitemap for each newly added dynamic content is a good approach though.

    However, I'll pass this to the dev team to see what they'll say about it and get back to you as soon as I get feedback.

    0
  • Comment author
    Teresa Klusaw

    Hi,

    We heard back from the dev team. They think this is likely a Webflow issue and recommend contacting their support.

    I hope that helps!

    0
  • Comment author
    Alican Avcuoğlu

    Will do, thank you both for your help

    0

Please sign in to leave a comment.