robots.txt
<head>
section of the final HTML<Head>
component<head>
tags from Frontity. The <Head>
component provided by the frontity
package uses React Helmet internally, so you can work as if you were working with common HTML.<head>
, you just have to import Head
from frontity
and write inside <Head>
all the tags you want. Usually, you will want to import it at the index.js
of your theme, in order for it to be loaded on all your pages.<Head>
that will be rendered normally.robots.txt
robots.txt
file tells search engine crawlers which pages or files the crawler should or shouldn't request from your site.robots.txt
file at the root of your theme (next to the frontity.settings.js
file) and when you build and deploy your app for production, it will be automatically picked up by the Frontity server and served at https://your-site-url.xyx/robots.txt
.