Unmanaged setup

This guide is for enterprise customers who are using the "unmanaged" setup (also known as crawler) to onboard onto c/side. The unmanaged setup is ideal for situations where you cannot or prefer not to make code changes to your site.

Important: To meet PCI DSS v4.0.1 requirement 4.6.3, you must have a mechanism in place to prevent unauthorized scripts from loading. You can do this by using a Content Security Policy (CSP) header with our unmanaged service. For your convenience, c/side offers a CSP reporting endpoint as part of the enterprise plan. More documentation on how to setup CSP here.

Step 1: Add the top level domain you wish to crawl

The domain you enter must have a valid DNS record for the top level.

Note: You can enable the wildcard option to automatically crawl any subdomain links found on the top level site. By default, the crawler will follow links up to 20 levels deep.

Step 2: Add additional subdomains

You can manually add specific subdomains of your site. The crawler will directly access these subdomains and crawl from there up to 20 levels deep.

Step 3: Add specific URLs that must be crawled

In the sidebar, click on "Domains". Then expand the domain you wish to configure and look for "Page Settings". Here you can specify individual pages that must be included in the crawl.

Optional: Adding Cookies, LocalStorage, and SessionStorage values

Some pages may require users to be logged in or have items in their cart. These states are typically managed through Cookies, LocalStorage, or SessionStorage values.

You can add these values to the c/side crawler at two levels:

• Page level: Through Page Settings (takes priority)
• Global level: Through Global Settings (applies to all pages unless overridden)

This allows the crawler to access protected or state-dependent content.

Sitemap Settings

You can provide your website's sitemap to the crawler. This will ensure all pages listed in the sitemap are automatically crawled, providing comprehensive coverage of your site.